-- Logs begin at Tue 2022-09-06 15:35:06 UTC, end at Tue 2022-09-06 16:41:03 UTC. -- Sep 06 16:01:47.825105 dv-cinder-852210-10-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 06 16:01:47.834269 dv-cinder-852210-10-iscsi devstack@keystone.service[81686]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 06 16:01:47.836266 dv-cinder-852210-10-iscsi devstack@keystone.service[81686]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Sep 6 16:01:47 2022] *** Sep 06 16:01:47.836266 dv-cinder-852210-10-iscsi devstack@keystone.service[81686]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 06 16:01:47.836266 dv-cinder-852210-10-iscsi devstack@keystone.service[81686]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 06 16:01:47.836266 dv-cinder-852210-10-iscsi devstack@keystone.service[81686]: nodename: dv-cinder-852210-10-iscsi Sep 06 16:01:47.836266 dv-cinder-852210-10-iscsi devstack@keystone.service[81686]: machine: x86_64 Sep 06 16:01:47.836266 dv-cinder-852210-10-iscsi devstack@keystone.service[81686]: clock source: unix Sep 06 16:01:47.836266 dv-cinder-852210-10-iscsi devstack@keystone.service[81686]: pcre jit disabled Sep 06 16:01:47.836266 dv-cinder-852210-10-iscsi devstack@keystone.service[81686]: detected number of CPU cores: 4 Sep 06 16:01:47.836266 dv-cinder-852210-10-iscsi devstack@keystone.service[81686]: current working directory: / Sep 06 16:01:47.836266 dv-cinder-852210-10-iscsi devstack@keystone.service[81686]: detected binary path: /usr/bin/uwsgi-core Sep 06 16:01:47.836266 dv-cinder-852210-10-iscsi devstack@keystone.service[81686]: your processes number limit is 47680 Sep 06 16:01:47.836266 dv-cinder-852210-10-iscsi devstack@keystone.service[81686]: your memory page size is 4096 bytes Sep 06 16:01:47.836266 dv-cinder-852210-10-iscsi devstack@keystone.service[81686]: detected max file descriptor number: 2048 Sep 06 16:01:47.836964 dv-cinder-852210-10-iscsi devstack@keystone.service[81686]: lock engine: pthread robust mutexes Sep 06 16:01:47.836964 dv-cinder-852210-10-iscsi devstack@keystone.service[81686]: thunder lock: enabled Sep 06 16:01:47.836964 dv-cinder-852210-10-iscsi devstack@keystone.service[81686]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 06 16:01:47.836964 dv-cinder-852210-10-iscsi devstack@keystone.service[81686]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 06 16:01:47.874988 dv-cinder-852210-10-iscsi devstack@keystone.service[81686]: Python main interpreter initialized at 0x56225d29d9c0 Sep 06 16:01:47.874988 dv-cinder-852210-10-iscsi devstack@keystone.service[81686]: python threads support enabled Sep 06 16:01:47.874988 dv-cinder-852210-10-iscsi devstack@keystone.service[81686]: your server socket listen backlog is limited to 100 connections Sep 06 16:01:47.874988 dv-cinder-852210-10-iscsi devstack@keystone.service[81686]: your mercy for graceful operations on workers is 90 seconds Sep 06 16:01:47.875337 dv-cinder-852210-10-iscsi devstack@keystone.service[81686]: mapped 403077 bytes (393 KB) for 2 cores Sep 06 16:01:47.875414 dv-cinder-852210-10-iscsi devstack@keystone.service[81686]: *** Operational MODE: preforking *** Sep 06 16:01:47.875658 dv-cinder-852210-10-iscsi devstack@keystone.service[81686]: *** uWSGI is running in multiple interpreter mode *** Sep 06 16:01:47.875658 dv-cinder-852210-10-iscsi devstack@keystone.service[81686]: spawned uWSGI master process (pid: 81686) Sep 06 16:01:47.875863 dv-cinder-852210-10-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 06 16:01:47.880604 dv-cinder-852210-10-iscsi devstack@keystone.service[81686]: spawned uWSGI worker 1 (pid: 81697, cores: 1) Sep 06 16:01:47.880604 dv-cinder-852210-10-iscsi devstack@keystone.service[81686]: spawned uWSGI worker 2 (pid: 81698, cores: 1) Sep 06 16:01:47.880604 dv-cinder-852210-10-iscsi devstack@keystone.service[81686]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 06 16:01:52.101051 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 06 16:01:52.101051 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: from cryptography.utils import int_from_bytes Sep 06 16:01:52.101900 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 06 16:01:52.101900 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: from cryptography.utils import int_from_bytes Sep 06 16:01:52.103186 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 06 16:01:52.103186 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: from cryptography.utils import int_from_bytes Sep 06 16:01:52.103685 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 06 16:01:52.103685 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: from cryptography.utils import int_from_bytes Sep 06 16:01:52.151221 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 06 16:01:52.151415 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 06 16:01:52.151581 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] command line args: [] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 06 16:01:52.151841 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 06 16:01:52.151993 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ================================================================================ {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 06 16:01:52.152262 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] admin_token = **** {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.152390 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 06 16:01:52.152479 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 06 16:01:52.152545 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] config_dir = [] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.152668 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] command line args: [] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 06 16:01:52.152775 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.152854 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] config_source = [] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.152924 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 06 16:01:52.152987 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] debug = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.153116 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ================================================================================ {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 06 16:01:52.153198 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.154856 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] admin_token = **** {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.154935 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.155000 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] config_dir = [] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.155067 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.155126 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.155126 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] config_source = [] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.155232 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.155296 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] debug = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.155358 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.155420 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.155500 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] list_limit = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.155557 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.155623 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] log_config_append = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.155680 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.155739 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.155825 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.155886 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] log_dir = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.155886 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] log_file = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.156002 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.156065 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.156126 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] list_limit = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.156126 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] log_config_append = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.156230 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.156291 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.156356 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.156414 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] log_dir = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.156475 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.156539 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] log_file = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.156600 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.156665 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.156722 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.156805 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.156876 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.156935 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.157009 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.157077 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.157136 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.157196 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.157252 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.157312 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.157505 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.157587 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.157646 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.157707 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.157894 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.158097 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.158179 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.158314 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.158314 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.158491 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.158633 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.158817 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.158936 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.43/identity {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.159032 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] publish_errors = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.159104 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.159233 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.159308 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.159434 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.159630 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.159749 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.43/identity {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.159831 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.159919 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] publish_errors = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.160095 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.160215 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.160293 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] standard_threads = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.160444 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.160532 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.160609 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.160785 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.160914 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.160991 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.161084 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.161159 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] use_journal = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.161230 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] standard_threads = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.161304 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] use_json = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.161452 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] use_stderr = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.161572 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.161704 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] use_syslog = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.161780 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.161849 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.161918 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.161985 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] use_journal = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.162052 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.162206 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] use_json = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.162337 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.162420 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] use_stderr = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.162891 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] use_syslog = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.162970 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.163123 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.163255 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 16:01:52.163335 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.163412 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.163592 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.163713 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.163813 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.163915 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.164074 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.164205 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.164284 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.164354 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.164554 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.164689 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.164819 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.164902 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.165056 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.165177 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.165252 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.165341 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.165526 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.165613 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.165762 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.165842 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.165997 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.166087 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.166232 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.166315 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.166467 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.166547 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.166713 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.166802 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.166953 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.167037 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.167185 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.167270 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.167433 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.167596 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.167669 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.167744 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.167925 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.168005 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.168157 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.168278 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.168356 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.168424 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.168603 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.168688 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.168855 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.168937 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.169092 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.169172 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.169338 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.169418 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.169582 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.169668 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.169811 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.169898 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.170026 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.170103 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.170270 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.170348 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.170516 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.170597 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.170747 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.170858 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.170980 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.171058 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.171205 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.171327 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.171439 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.171554 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.171672 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.171782 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.171889 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.171999 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.172107 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.172214 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.172330 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.172444 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.172591 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.172701 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.172918 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.173032 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.173224 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.173336 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.173480 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.173592 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.173701 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.173815 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.173933 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.174073 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.174189 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.174299 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.174409 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.174516 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.174622 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.174722 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.174814 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.174906 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.174998 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.175090 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.175191 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.175286 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.175377 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.175469 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.175561 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.175656 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.175740 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.175811 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.175929 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.176069 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.176190 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.176311 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.176427 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.176528 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.176643 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.176766 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.176888 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.177007 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.177125 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.177234 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.177349 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.177463 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.177565 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.177682 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.177793 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.177913 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.178024 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.178140 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.178243 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.178444 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.178556 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.178678 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.178790 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.178904 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.179014 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.179115 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.179231 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.179351 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.179462 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.179571 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.179689 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.179800 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.179910 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.180027 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.180140 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.180250 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.180360 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.180475 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.180587 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.180703 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.180862 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.180974 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.181094 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.181207 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.181322 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.181443 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.181559 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.181670 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.181779 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.181890 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.182005 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.182122 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.182233 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.182342 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.182453 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.182562 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.182671 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.182787 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.182898 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.183007 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.183117 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.183226 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.183335 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.183453 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.183565 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.183674 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.183783 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.183892 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.184001 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.184186 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.184299 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.184408 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.184525 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.184636 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.184768 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.184768 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.185000 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.185111 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.185220 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.185330 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.185439 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.185549 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.185668 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.185780 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.185895 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.186007 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.186119 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.186229 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.186346 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.186346 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.186554 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.186665 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.186774 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.186883 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.187000 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.187119 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.187230 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] auth.external = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.187230 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.187450 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.187450 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.187698 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.187812 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.187931 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.187931 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] auth.password = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.188140 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.188250 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] auth.token = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.188368 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] auth.external = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.188480 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.188597 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.188710 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.188863 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.188976 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.189086 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.189197 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.189318 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] auth.password = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.189431 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.189542 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] auth.token = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.189653 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.189764 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.189874 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.189992 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.190104 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] credential.caching = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.190214 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.190325 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.190435 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.190547 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.190670 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.190785 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.190896 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.191006 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.191116 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.191226 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.191343 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] credential.caching = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.191455 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.191565 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.191681 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.191793 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.191905 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.192023 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.192135 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.192246 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.192356 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.192466 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.192577 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.192694 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.192820 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.192933 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.193044 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.193154 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.193296 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.193419 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.193533 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.193643 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.193754 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] federation.caching = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.193864 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.193975 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.194093 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.194212 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.194324 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.194434 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.194544 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.194655 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.194773 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.194885 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.194997 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] federation.caching = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.195113 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.195225 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.195343 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.195455 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.195565 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.195682 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.195794 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.195904 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.196022 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.196135 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.196246 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.196356 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.196466 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] identity.caching = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.196576 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.196694 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.196863 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.196976 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.197088 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.197200 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.197310 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.197510 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.197626 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.197737 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.197847 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] identity.caching = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.197963 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.198076 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.198195 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.198307 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.198418 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.198528 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.198644 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.198755 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.198865 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.198976 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.199086 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.199197 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.199307 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.199417 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.199534 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.199646 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.199757 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.199866 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.199976 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.200085 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.200203 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.200316 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.200427 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.200537 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.200647 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.200775 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.200901 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.201021 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.201132 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.201243 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.201354 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.201464 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.201582 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.201694 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.201804 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.201804 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.202011 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.202121 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.202240 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.202353 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.202528 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.202639 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.202749 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.202860 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.202978 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.203090 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.203200 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.203200 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.203408 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.203408 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.203627 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.203739 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.203848 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.203958 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.204068 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.204177 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.204295 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.204407 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.204520 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.204630 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.204739 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.204911 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.205024 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.205134 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.205245 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.205355 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.205466 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.205584 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.205696 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.205806 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.205916 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.206027 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.206136 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.206136 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.206353 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.206469 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.206581 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.206691 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.206801 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.206918 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.207031 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.207141 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.207250 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.207360 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.207470 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.207593 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.207706 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.207816 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.207926 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.208036 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.208146 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.208146 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.208363 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.208474 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.208962 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.209080 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.209191 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.209309 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.209421 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.209596 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.209718 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.209829 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.209940 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.210059 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.210180 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.210290 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.210402 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.210513 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.210623 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.210743 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.210856 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.210967 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.211078 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.211189 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.211305 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.211428 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.211541 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.211651 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.211762 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.211873 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.211983 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.212101 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.212214 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.212325 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.212435 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.212546 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.212656 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.212794 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.212909 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.213020 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.213131 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.213244 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.213353 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.213476 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.213589 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.213717 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.213849 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.213960 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.214070 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.214189 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.214301 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.214412 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.214521 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.214631 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.214741 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.214861 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.214974 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.215085 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.215195 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.215305 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.215415 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.215538 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.215651 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.215761 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.215871 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.215982 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.216092 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.216210 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.216322 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.216432 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.216542 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.216652 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.216808 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.216938 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] resource.caching = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.217053 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.217171 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.217283 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.217393 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.217504 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.217626 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.217738 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.217848 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.217959 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.218070 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.218180 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.218299 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.218412 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] resource.caching = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.218523 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.218634 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.218744 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.218854 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.218972 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.219082 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.219199 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] role.caching = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.219310 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.219420 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] role.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.219605 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.219718 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.219829 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.219940 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.220050 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.220161 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.220280 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.220392 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.220502 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.220612 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.220723 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] role.caching = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.220850 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.220973 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] role.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.221086 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.221197 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.221307 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.221417 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.221526 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.221644 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.221757 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.221860 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.221969 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.222078 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.222189 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.222308 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.222421 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.222531 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.222642 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.222752 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.222862 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.222980 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.223092 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.223203 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.223314 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.223425 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.223535 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.223653 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.223765 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.223875 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.223985 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.224095 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.224205 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.224323 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.224436 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.224546 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.224658 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.224814 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.224940 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.224940 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.225177 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.225289 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.225401 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.225510 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.225631 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.225744 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.225854 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.225966 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.226076 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.226187 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.226305 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.226417 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.226529 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.226639 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.226749 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.226859 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.226976 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.227089 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] token.caching = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.227199 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.227308 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.227418 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.227529 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.227651 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.227763 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.227873 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.227983 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.228093 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.228202 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.228320 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.228432 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.228544 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.228654 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.228800 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] token.caching = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.228913 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.229037 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.229141 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.229251 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.229362 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.229473 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.229584 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.229702 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.229814 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.229924 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.230035 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.230145 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.230261 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.230380 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.230492 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.230611 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.230733 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.230846 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.230956 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.231067 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.231185 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.connection = **** {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.231297 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.231408 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.231518 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.231629 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.231741 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.231860 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.231973 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.232082 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.232193 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.232311 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.232431 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.232543 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.232655 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.232829 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.232944 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.connection = **** {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.233056 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.233166 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.233276 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.233386 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.233508 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.233621 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.233731 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.233843 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.233953 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.234063 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.234184 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.234296 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.234407 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.234517 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.234628 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.234738 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.234858 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.234970 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.235080 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.235190 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.235300 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.235411 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 06 16:01:52.235529 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.235641 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.235814 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.236048 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.236290 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.236521 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.236778 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.237013 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 16:01:52.237186 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 06 16:01:52.264217 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 16:01:52.264444 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 16:01:52.264669 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 16:01:52.265154 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 16:01:52.265385 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 16:01:52.265605 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 16:01:52.274353 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 16:01:52.274615 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 16:01:52.301118 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 16:01:52.301336 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 16:01:52.301745 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 16:01:52.301974 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 16:01:52.354985 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 16:01:52.355139 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 16:01:52.406431 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 16:01:52.406638 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 16:01:52.408781 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 16:01:52.409042 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 16:01:52.565414 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 16:01:52.565627 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 16:01:52.565839 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 16:01:52.566055 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 16:01:52.566210 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 16:01:52.566298 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 16:01:52.566373 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 16:01:52.566447 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 16:01:52.566519 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 16:01:52.566684 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 16:01:52.566906 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 16:01:52.567111 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 16:01:52.589642 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 16:01:52.589853 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 16:01:52.590070 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 16:01:52.590181 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 16:01:52.590314 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 16:01:52.590392 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 16:01:52.590453 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 16:01:52.590641 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 16:01:52.590780 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 16:01:52.590913 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 16:01:52.590992 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 16:01:52.591054 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 16:01:52.591264 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 16:01:52.591483 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 16:01:52.602167 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 16:01:52.602439 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 16:01:52.614829 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 16:01:52.615027 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 16:01:52.615188 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 16:01:52.615439 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 16:01:52.615652 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 16:01:52.615903 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 16:01:52.616072 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 16:01:52.616200 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 16:01:52.616294 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 16:01:52.616402 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 16:01:52.616402 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 16:01:52.616529 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 16:01:52.616782 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 16:01:52.616990 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 16:01:52.632803 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.633039 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 06 16:01:52.633162 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.633383 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.633507 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 06 16:01:52.633645 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 06 16:01:52.633729 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.633791 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.633911 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 06 16:01:52.634054 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.634151 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.634342 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.634468 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.634601 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.634679 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.634879 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.652799 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.652799 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.652799 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.653176 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.653254 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.653349 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.653349 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.653349 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.653541 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 16:01:52.663295 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 16:01:52.663468 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 16:01:52.663544 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.663544 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.663544 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.663544 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.663544 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.663544 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.663544 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.663897 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.663897 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.663897 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.664057 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 16:01:52.664057 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.664057 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.664057 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.664057 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.664057 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.664057 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.664406 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.664406 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.664406 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.664406 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.680766 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.682257 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 16:01:52.682552 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.687217 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 16:01:52.687472 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.687780 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.692504 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.695441 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 16:01:52.696186 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 16:01:52.696278 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.696377 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.702289 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 16:01:52.702578 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.702855 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.704055 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 16:01:52.704363 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.704657 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.704973 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.705257 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.705536 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.705812 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.706115 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.711293 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 16:01:52.711736 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.718826 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 16:01:52.719122 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.719406 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.719696 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.719974 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.720247 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.720515 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.720792 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.723877 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.724166 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.728066 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 16:01:52.731329 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.731629 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.735855 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 16:01:52.736222 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.736494 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.738994 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 16:01:52.740346 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 16:01:52.742534 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.742836 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.743155 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.743422 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.743631 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.743778 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.743921 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.744002 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.748091 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 16:01:52.751377 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 16:01:52.754603 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.754910 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.755200 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.755483 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.755761 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.756049 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.759785 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 16:01:52.760075 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.760342 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.760606 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.767492 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.769048 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.770582 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.770858 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.771707 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 16:01:52.772048 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.772316 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.772581 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.773603 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 16:01:52.773905 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.774257 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.774540 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.774938 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.779411 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.780985 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.782490 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.782777 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.785543 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 16:01:52.785828 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.786119 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.786261 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 16:01:52.786399 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.786562 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.786646 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.786790 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.786915 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.787310 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.787623 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.787902 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.798014 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 16:01:52.798297 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.798579 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.798855 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.799229 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.799515 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.799792 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.803408 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 16:01:52.806341 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 16:01:52.809257 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.810681 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.812209 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 16:01:52.812490 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.812914 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.815127 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 16:01:52.818109 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 16:01:52.819297 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.820779 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 16:01:52.821028 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.822488 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.823668 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.823953 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 16:01:52.824077 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.824214 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.824354 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.824609 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.824732 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.830998 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.832453 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 16:01:52.832576 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 16:01:52.832852 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.833134 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.835773 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.836164 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.836449 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.836721 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.840420 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 16:01:52.840703 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.841014 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.841296 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.841571 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.841862 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.842133 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.842408 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.842682 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.842972 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.843243 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.843494 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.843760 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.844019 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.844470 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 16:01:52.844779 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.845072 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.851941 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 16:01:52.852220 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.852496 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.852791 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.853078 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.853333 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.853602 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.853874 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.854151 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.854434 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.854709 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.854966 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.855224 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.855474 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 16:01:52.878196 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x56225d29d9c0 pid: 81697 (default app) Sep 06 16:01:52.881517 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e271a0-66d3-422c-960a-3b244e46ed83 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:52.881770 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e271a0-66d3-422c-960a-3b244e46ed83 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:52.882070 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e271a0-66d3-422c-960a-3b244e46ed83 None None] PATH_INFO: `/v3` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:52.886247 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1/1] 10.222.0.43 () {50 vars in 791 bytes} [Tue Sep 6 16:01:52 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:52.889146 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x56225d29d9c0 pid: 81698 (default app) Sep 06 16:01:59.484207 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fbf62e-521f-4410-bf50-2f9225689ac0 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:59.485116 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fbf62e-521f-4410-bf50-2f9225689ac0 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:59.485116 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fbf62e-521f-4410-bf50-2f9225689ac0 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:59.487262 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 2/2] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:01:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:01:59.499749 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429908e8-def1-45c6-afa2-789c84127a34 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:59.500221 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429908e8-def1-45c6-afa2-789c84127a34 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:59.500360 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429908e8-def1-45c6-afa2-789c84127a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:59.579895 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: INFO dbcounter [None req-429908e8-def1-45c6-afa2-789c84127a34 None None] Registered counter for database keystone Sep 06 16:01:59.601326 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG oslo_db.sqlalchemy.engines [None req-429908e8-def1-45c6-afa2-789c84127a34 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81698) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 06 16:01:59.612790 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writer thread running {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 06 16:01:59.796987 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG passlib.handlers.bcrypt [None req-429908e8-def1-45c6-afa2-789c84127a34 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81698) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 06 16:01:59.797377 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG passlib.handlers.bcrypt [None req-429908e8-def1-45c6-afa2-789c84127a34 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81698) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 06 16:01:59.856418 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-429908e8-def1-45c6-afa2-789c84127a34 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:01:59.857997 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-429908e8-def1-45c6-afa2-789c84127a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:00.001794 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1/3] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:01:59 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 507 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 06 16:02:00.509060 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fbf62e-521f-4410-bf50-2f9225689ac0 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:00.587142 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: INFO dbcounter [None req-f4fbf62e-521f-4410-bf50-2f9225689ac0 None None] Registered counter for database keystone Sep 06 16:02:00.612069 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG oslo_db.sqlalchemy.engines [None req-f4fbf62e-521f-4410-bf50-2f9225689ac0 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81697) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 06 16:02:00.620237 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writer thread running {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 06 16:02:00.739366 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fbf62e-521f-4410-bf50-2f9225689ac0 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:00.744950 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9205ab1-789a-4f30-a0f8-664140ea8756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:00.745575 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9205ab1-789a-4f30-a0f8-664140ea8756 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:00.745856 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9205ab1-789a-4f30-a0f8-664140ea8756 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:00.746129 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9205ab1-789a-4f30-a0f8-664140ea8756 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:00.755424 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9205ab1-789a-4f30-a0f8-664140ea8756 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:00.806084 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: WARNING keystone.common.rbac_enforcer.enforcer [None req-e9205ab1-789a-4f30-a0f8-664140ea8756 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 06 16:02:00.806427 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9205ab1-789a-4f30-a0f8-664140ea8756 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:00.816520 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 3/4] 10.222.0.43 () {58 vars in 1324 bytes} [Tue Sep 6 16:02:00 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 16:02:00.823643 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429908e8-def1-45c6-afa2-789c84127a34 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:00.844948 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429908e8-def1-45c6-afa2-789c84127a34 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:00.852658 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdcd382-2f0a-4ae9-991a-af3091ea02bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:00.854487 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdcd382-2f0a-4ae9-991a-af3091ea02bb admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:00.854932 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdcd382-2f0a-4ae9-991a-af3091ea02bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:00.855362 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdcd382-2f0a-4ae9-991a-af3091ea02bb admin admin] PATH_INFO: `/v3/projects/0ad262a5a8a84066b8eba54c7ea32d39` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:00.857972 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdcd382-2f0a-4ae9-991a-af3091ea02bb admin admin] RBAC: Authorizing `identity:get_project(project_id=0ad262a5a8a84066b8eba54c7ea32d39)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:00.913918 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: WARNING keystone.common.rbac_enforcer.enforcer [None req-bfdcd382-2f0a-4ae9-991a-af3091ea02bb admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 06 16:02:00.914233 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdcd382-2f0a-4ae9-991a-af3091ea02bb admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:00.918877 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 2/5] 10.222.0.43 () {56 vars in 1207 bytes} [Tue Sep 6 16:02:00 2022] GET /identity/v3/projects/0ad262a5a8a84066b8eba54c7ea32d39 => generated 344 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:02.800939 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e963445-43d5-48a7-8c88-82236635f8c5 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:02.801677 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e963445-43d5-48a7-8c88-82236635f8c5 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:02.801677 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e963445-43d5-48a7-8c88-82236635f8c5 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:02.805635 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 4/6] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:02:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 16:02:02.811696 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f7992-7ac6-4d19-b1c7-1dc6129d8df8 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:02.812112 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f7992-7ac6-4d19-b1c7-1dc6129d8df8 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:02.812195 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f7992-7ac6-4d19-b1c7-1dc6129d8df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:02.835016 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-fe3f7992-7ac6-4d19-b1c7-1dc6129d8df8 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:02.836550 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-fe3f7992-7ac6-4d19-b1c7-1dc6129d8df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:02.846898 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 3/7] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:02 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 06 16:02:03.119154 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e963445-43d5-48a7-8c88-82236635f8c5 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:03.133622 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e963445-43d5-48a7-8c88-82236635f8c5 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:03.139055 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d28585-7c1b-4e4a-9cd7-274585e6d9e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:03.139654 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d28585-7c1b-4e4a-9cd7-274585e6d9e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:03.139932 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d28585-7c1b-4e4a-9cd7-274585e6d9e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:03.140203 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d28585-7c1b-4e4a-9cd7-274585e6d9e5 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:03.148794 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d28585-7c1b-4e4a-9cd7-274585e6d9e5 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:03.150736 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d28585-7c1b-4e4a-9cd7-274585e6d9e5 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:03.159624 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 5/8] 10.222.0.43 () {58 vars in 1324 bytes} [Tue Sep 6 16:02:03 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 16:02:03.165501 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3f7992-7ac6-4d19-b1c7-1dc6129d8df8 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:03.178325 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3f7992-7ac6-4d19-b1c7-1dc6129d8df8 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:03.185549 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2a96a4-e630-4e0b-bacf-8091c2ab771c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:03.186192 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a96a4-e630-4e0b-bacf-8091c2ab771c admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:03.186468 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a96a4-e630-4e0b-bacf-8091c2ab771c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:03.186740 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a96a4-e630-4e0b-bacf-8091c2ab771c admin admin] PATH_INFO: `/v3/users/824c9880f41b4962b83243abf75b8162` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:03.188180 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2a96a4-e630-4e0b-bacf-8091c2ab771c admin admin] RBAC: Authorizing `identity:get_user(user_id=824c9880f41b4962b83243abf75b8162)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:03.190104 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2a96a4-e630-4e0b-bacf-8091c2ab771c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:03.191670 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 4/9] 10.222.0.43 () {56 vars in 1198 bytes} [Tue Sep 6 16:02:03 2022] GET /identity/v3/users/824c9880f41b4962b83243abf75b8162 => generated 247 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:05.184377 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b91ea34-92f0-48d2-9aaf-82049198371c None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:05.184967 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b91ea34-92f0-48d2-9aaf-82049198371c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:05.184967 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b91ea34-92f0-48d2-9aaf-82049198371c None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:05.186336 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 6/10] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:02:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 16:02:05.191723 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d154de4-6c2e-42ad-9b89-e09d04a8baf4 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:05.192153 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d154de4-6c2e-42ad-9b89-e09d04a8baf4 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:05.192245 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d154de4-6c2e-42ad-9b89-e09d04a8baf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:05.211853 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-8d154de4-6c2e-42ad-9b89-e09d04a8baf4 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:05.213094 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-8d154de4-6c2e-42ad-9b89-e09d04a8baf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:05.215473 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46fd657-8dc9-421d-bb62-a8132c4997cc None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:05.215766 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46fd657-8dc9-421d-bb62-a8132c4997cc None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:05.216045 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46fd657-8dc9-421d-bb62-a8132c4997cc None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:05.217401 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 7/11] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:02:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 16:02:05.222408 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e20c4f1-7fc6-406f-80c9-e6fb5fa7e873 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:05.223000 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e20c4f1-7fc6-406f-80c9-e6fb5fa7e873 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:05.223000 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e20c4f1-7fc6-406f-80c9-e6fb5fa7e873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:05.265090 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG passlib.handlers.bcrypt [None req-5e20c4f1-7fc6-406f-80c9-e6fb5fa7e873 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81697) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 06 16:02:05.265478 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG passlib.handlers.bcrypt [None req-5e20c4f1-7fc6-406f-80c9-e6fb5fa7e873 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81697) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 06 16:02:05.293963 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 5/12] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:02:05.313951 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-5e20c4f1-7fc6-406f-80c9-e6fb5fa7e873 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:05.315553 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5e20c4f1-7fc6-406f-80c9-e6fb5fa7e873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:05.416776 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 8/13] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 16:02:05.558417 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d154de4-6c2e-42ad-9b89-e09d04a8baf4 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:05.582906 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:05.666725 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d154de4-6c2e-42ad-9b89-e09d04a8baf4 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:05.681769 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e20c4f1-7fc6-406f-80c9-e6fb5fa7e873 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:05.702491 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be6e315-52e7-4770-a68e-0e27b174e0c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:05.703143 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be6e315-52e7-4770-a68e-0e27b174e0c8 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:05.703424 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be6e315-52e7-4770-a68e-0e27b174e0c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:05.703696 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be6e315-52e7-4770-a68e-0e27b174e0c8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:05.715464 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be6e315-52e7-4770-a68e-0e27b174e0c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:02:05.724362 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-8be6e315-52e7-4770-a68e-0e27b174e0c8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:02:05.724362 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:05.724362 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:05.724362 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:05.724362 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:05.724362 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:05.724362 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:05.724362 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:05.724362 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:05.724362 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:05.724362 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:05.724362 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:05.724362 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:02:05.724362 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:02:05.724362 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:02:05.724362 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:02:05.725474 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:02:05.725474 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:02:05.725474 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:02:05.725474 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:02:05.725474 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:05.725474 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:05.725474 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:05.725474 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:05.725474 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:05.725474 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:05.725474 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:05.725474 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:05.725474 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:05.725474 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:05.725474 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:05.725474 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:05.726401 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:05.726401 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:05.726401 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:05.726401 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:05.726401 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:02:05.726401 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:02:05.726401 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:02:05.726401 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:02:05.726401 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:02:05.726401 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:02:05.726401 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:02:05.726401 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:02:05.726401 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 6/14] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 6 16:02:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:02:05.731367 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be6e315-52e7-4770-a68e-0e27b174e0c8 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:05.783600 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e20c4f1-7fc6-406f-80c9-e6fb5fa7e873 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:05.785895 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:05.813298 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:05.823980 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da40c24-bfa4-4eaa-ab45-c8d45cfe8755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:05.824924 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da40c24-bfa4-4eaa-ab45-c8d45cfe8755 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:05.825201 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da40c24-bfa4-4eaa-ab45-c8d45cfe8755 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:05.825465 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da40c24-bfa4-4eaa-ab45-c8d45cfe8755 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:05.831948 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be6e315-52e7-4770-a68e-0e27b174e0c8 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:05.869157 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffd2801-f8cd-4c38-abf7-f8aafedb511b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:05.869793 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffd2801-f8cd-4c38-abf7-f8aafedb511b None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:05.870081 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffd2801-f8cd-4c38-abf7-f8aafedb511b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:05.870353 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffd2801-f8cd-4c38-abf7-f8aafedb511b None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:05.871043 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffd2801-f8cd-4c38-abf7-f8aafedb511b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:05.871349 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffd2801-f8cd-4c38-abf7-f8aafedb511b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:05.872798 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffd2801-f8cd-4c38-abf7-f8aafedb511b None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:05.885334 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 7/15] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 6 16:02:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:02:05.891059 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffd2801-f8cd-4c38-abf7-f8aafedb511b None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:05.899895 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da40c24-bfa4-4eaa-ab45-c8d45cfe8755 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:05.901638 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da40c24-bfa4-4eaa-ab45-c8d45cfe8755 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:05.990516 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffd2801-f8cd-4c38-abf7-f8aafedb511b None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:06.008856 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 9/16] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 6 16:02:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 06 16:02:06.015899 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da40c24-bfa4-4eaa-ab45-c8d45cfe8755 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:06.019019 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:06.026259 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1936d4e3-bbba-4d13-b083-6152f550546f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:06.026904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1936d4e3-bbba-4d13-b083-6152f550546f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:06.027177 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1936d4e3-bbba-4d13-b083-6152f550546f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:06.027462 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1936d4e3-bbba-4d13-b083-6152f550546f None admin] PATH_INFO: `/v3/users/824c9880f41b4962b83243abf75b8162` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:06.028674 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1936d4e3-bbba-4d13-b083-6152f550546f None admin] RBAC: Authorizing `identity:get_user(user_id=824c9880f41b4962b83243abf75b8162)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:06.030186 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1936d4e3-bbba-4d13-b083-6152f550546f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:06.031783 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 8/17] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 6 16:02:05 2022] GET /identity/v3/users/824c9880f41b4962b83243abf75b8162 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:06.037540 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1936d4e3-bbba-4d13-b083-6152f550546f None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:06.052279 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:06.113927 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da40c24-bfa4-4eaa-ab45-c8d45cfe8755 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:06.135677 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1936d4e3-bbba-4d13-b083-6152f550546f None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:06.147657 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e135a91-4f64-4688-a0b5-e5206bd95767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:06.148524 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e135a91-4f64-4688-a0b5-e5206bd95767 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:06.148837 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e135a91-4f64-4688-a0b5-e5206bd95767 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:06.149171 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e135a91-4f64-4688-a0b5-e5206bd95767 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:06.167851 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e135a91-4f64-4688-a0b5-e5206bd95767 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:06.169369 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e135a91-4f64-4688-a0b5-e5206bd95767 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:06.174834 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7357fd4-701f-4094-b739-c019851315cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:06.175507 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7357fd4-701f-4094-b739-c019851315cf None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:06.175819 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7357fd4-701f-4094-b739-c019851315cf None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:06.176115 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7357fd4-701f-4094-b739-c019851315cf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:06.179382 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7357fd4-701f-4094-b739-c019851315cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:06.179382 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7357fd4-701f-4094-b739-c019851315cf None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:06.180294 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 9/18] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:02:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:06.186165 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7357fd4-701f-4094-b739-c019851315cf None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:06.189824 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-8e135a91-4f64-4688-a0b5-e5206bd95767 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:02:06.189824 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:06.189824 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:06.189824 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:06.189824 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:06.189824 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:06.189824 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:06.189824 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:06.189824 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:06.189824 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:06.189824 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:06.189824 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:06.189824 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:02:06.189824 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:02:06.189824 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:02:06.189824 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:02:06.191039 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:06.191039 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:06.191039 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:06.191039 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:06.191039 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:06.191039 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:06.191039 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:06.191039 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:06.191039 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:06.191039 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:06.191039 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:06.191039 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:06.191039 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:06.191039 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:06.191039 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:06.191039 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:06.191039 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:02:06.192159 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:02:06.192159 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:02:06.192159 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:02:06.192159 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 10/19] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:02:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:02:06.196648 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e135a91-4f64-4688-a0b5-e5206bd95767 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:06.256481 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:06.266984 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:06.287745 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7357fd4-701f-4094-b739-c019851315cf None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:06.300588 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e135a91-4f64-4688-a0b5-e5206bd95767 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:06.320668 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef284382-bf69-4d95-acc8-b38dc8588ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:06.321337 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef284382-bf69-4d95-acc8-b38dc8588ecb None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:06.321615 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef284382-bf69-4d95-acc8-b38dc8588ecb None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:06.321888 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef284382-bf69-4d95-acc8-b38dc8588ecb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:06.322622 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef284382-bf69-4d95-acc8-b38dc8588ecb None admin] RBAC: Adding query filter params (role.id=edaabdea8be84a7980fb2c651fe52f63, scope.domain.id=default, user.id=824c9880f41b4962b83243abf75b8162) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:06.322926 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef284382-bf69-4d95-acc8-b38dc8588ecb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:06.324388 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef284382-bf69-4d95-acc8-b38dc8588ecb None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:06.330964 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 10/20] 10.222.0.43 () {56 vars in 1322 bytes} [Tue Sep 6 16:02:06 2022] GET /identity/v3/role_assignments?user.id=824c9880f41b4962b83243abf75b8162&scope.domain.id=default&role.id=edaabdea8be84a7980fb2c651fe52f63 => generated 233 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:02:06.333996 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582a5357-6c1a-4f60-af3d-9915bf0968be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:06.334756 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a5357-6c1a-4f60-af3d-9915bf0968be None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:06.335038 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a5357-6c1a-4f60-af3d-9915bf0968be None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:06.335312 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a5357-6c1a-4f60-af3d-9915bf0968be None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:06.335983 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582a5357-6c1a-4f60-af3d-9915bf0968be None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:06.336287 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582a5357-6c1a-4f60-af3d-9915bf0968be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:06.337891 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582a5357-6c1a-4f60-af3d-9915bf0968be None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:06.359828 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 11/21] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:02:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:02:08.360631 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d0287-6db8-4ac7-a4b5-85441b61f713 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:08.361162 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d0287-6db8-4ac7-a4b5-85441b61f713 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:08.361258 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d0287-6db8-4ac7-a4b5-85441b61f713 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:08.362517 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 11/22] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:02:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:02:08.367969 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2be815-4e16-4dcf-8a3d-cc491d2a5589 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:08.368379 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2be815-4e16-4dcf-8a3d-cc491d2a5589 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:08.368580 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2be815-4e16-4dcf-8a3d-cc491d2a5589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:08.394348 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-ad2be815-4e16-4dcf-8a3d-cc491d2a5589 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:08.395545 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ad2be815-4e16-4dcf-8a3d-cc491d2a5589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:08.458272 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72efcbbe-3051-42b1-9e91-6be0afa3b1bb None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:08.458876 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72efcbbe-3051-42b1-9e91-6be0afa3b1bb None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:08.459541 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72efcbbe-3051-42b1-9e91-6be0afa3b1bb None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:08.463167 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 12/23] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:02:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 16:02:08.474138 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed939960-c1d3-4b3c-9a94-64609f3cab85 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:08.474649 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed939960-c1d3-4b3c-9a94-64609f3cab85 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:08.475218 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed939960-c1d3-4b3c-9a94-64609f3cab85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:08.489986 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 12/24] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:02:08.509936 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-ed939960-c1d3-4b3c-9a94-64609f3cab85 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:08.511517 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ed939960-c1d3-4b3c-9a94-64609f3cab85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:08.652768 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 13/25] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 16:02:08.860111 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2be815-4e16-4dcf-8a3d-cc491d2a5589 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:08.889330 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e4b779-bcd9-4719-8677-f7fc1466124e None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:08.889646 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e4b779-bcd9-4719-8677-f7fc1466124e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:08.889882 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e4b779-bcd9-4719-8677-f7fc1466124e None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:08.891233 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 14/26] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:02:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 16:02:08.907662 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71346ab-4a9f-4029-93d0-3cd13f712bd7 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:08.907913 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71346ab-4a9f-4029-93d0-3cd13f712bd7 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:08.908134 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71346ab-4a9f-4029-93d0-3cd13f712bd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:08.941595 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-f71346ab-4a9f-4029-93d0-3cd13f712bd7 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:08.942773 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f71346ab-4a9f-4029-93d0-3cd13f712bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:08.951560 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:09.060041 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2be815-4e16-4dcf-8a3d-cc491d2a5589 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:09.083314 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 15/27] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 06 16:02:09.113232 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600f8b0c-104c-46d7-8b56-b19a91b7ff97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:09.113880 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600f8b0c-104c-46d7-8b56-b19a91b7ff97 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:09.114164 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600f8b0c-104c-46d7-8b56-b19a91b7ff97 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:09.114447 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600f8b0c-104c-46d7-8b56-b19a91b7ff97 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:09.138594 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600f8b0c-104c-46d7-8b56-b19a91b7ff97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:02:09.145116 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-600f8b0c-104c-46d7-8b56-b19a91b7ff97 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:02:09.145116 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:09.145116 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:09.145116 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:09.145116 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:09.145116 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:09.145116 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:09.145116 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:09.145116 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:09.145116 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:09.145116 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:09.145116 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:09.145116 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:02:09.145116 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:02:09.145116 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:02:09.145116 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:02:09.145923 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:02:09.145923 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:02:09.145923 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:02:09.145923 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:02:09.145923 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:09.145923 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:09.145923 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:09.145923 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:09.145923 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:09.145923 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:09.145923 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:09.145923 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:09.145923 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:09.145923 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:09.145923 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:09.145923 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:09.146685 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:09.146685 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:09.146685 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:09.146685 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:09.146685 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:02:09.146685 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:02:09.146685 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:02:09.146685 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:02:09.146685 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:02:09.146685 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:02:09.146685 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:02:09.146685 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:02:09.146685 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 13/28] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 6 16:02:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:02:09.152247 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71346ab-4a9f-4029-93d0-3cd13f712bd7 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:09.155948 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:09.209214 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600f8b0c-104c-46d7-8b56-b19a91b7ff97 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:09.321588 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71346ab-4a9f-4029-93d0-3cd13f712bd7 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:09.386038 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b3ecb3-a27a-4266-87e9-a1778b4a0699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:09.386697 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b3ecb3-a27a-4266-87e9-a1778b4a0699 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:09.386945 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b3ecb3-a27a-4266-87e9-a1778b4a0699 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:09.387201 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b3ecb3-a27a-4266-87e9-a1778b4a0699 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:09.387866 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b3ecb3-a27a-4266-87e9-a1778b4a0699 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:09.388162 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b3ecb3-a27a-4266-87e9-a1778b4a0699 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:09.389770 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b3ecb3-a27a-4266-87e9-a1778b4a0699 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:09.406387 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 16/29] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 6 16:02:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:09.410893 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600f8b0c-104c-46d7-8b56-b19a91b7ff97 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:09.411634 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b3ecb3-a27a-4266-87e9-a1778b4a0699 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:09.424785 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:09.482856 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d81342d-f4e4-4cfb-92f5-5554b9e41235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:09.483422 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d81342d-f4e4-4cfb-92f5-5554b9e41235 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:09.483702 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d81342d-f4e4-4cfb-92f5-5554b9e41235 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:09.483975 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d81342d-f4e4-4cfb-92f5-5554b9e41235 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:09.494742 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d81342d-f4e4-4cfb-92f5-5554b9e41235 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:09.496233 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d81342d-f4e4-4cfb-92f5-5554b9e41235 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:09.512566 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-8d81342d-f4e4-4cfb-92f5-5554b9e41235 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:02:09.512566 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:09.512566 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:09.512566 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:09.512566 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:09.512566 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:09.512566 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:09.512566 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:09.512566 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:09.512566 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:09.512566 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:09.512566 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:09.512566 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:02:09.512566 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:02:09.512566 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:02:09.512566 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:02:09.513483 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:09.513483 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:09.513483 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:09.513483 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:09.513483 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:09.513483 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:09.513483 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:09.513483 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:09.513483 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:09.513483 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:09.513483 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:09.513483 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:09.513483 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:09.513483 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:09.513483 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:09.513483 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:09.513483 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:02:09.514259 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:02:09.514259 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:02:09.514259 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:02:09.514259 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 14/30] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:02:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:02:09.516123 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d81342d-f4e4-4cfb-92f5-5554b9e41235 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:09.533150 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:09.570876 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b3ecb3-a27a-4266-87e9-a1778b4a0699 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:09.606508 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe9199e-570d-4551-8c9b-eca6f9f44143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:09.607212 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe9199e-570d-4551-8c9b-eca6f9f44143 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:09.607558 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe9199e-570d-4551-8c9b-eca6f9f44143 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:09.607753 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe9199e-570d-4551-8c9b-eca6f9f44143 None admin] PATH_INFO: `/v3/users/824c9880f41b4962b83243abf75b8162` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:09.608962 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe9199e-570d-4551-8c9b-eca6f9f44143 None admin] RBAC: Authorizing `identity:get_user(user_id=824c9880f41b4962b83243abf75b8162)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:09.610666 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe9199e-570d-4551-8c9b-eca6f9f44143 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:09.612618 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 17/31] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 6 16:02:09 2022] GET /identity/v3/users/824c9880f41b4962b83243abf75b8162 => generated 247 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:02:09.613569 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd64eac-363b-4e07-8dfa-b21b48f12d2a None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:09.613849 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd64eac-363b-4e07-8dfa-b21b48f12d2a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:09.614118 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd64eac-363b-4e07-8dfa-b21b48f12d2a None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:09.615514 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 18/32] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:02:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:02:09.616225 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd64eac-363b-4e07-8dfa-b21b48f12d2a None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:09.629976 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d81342d-f4e4-4cfb-92f5-5554b9e41235 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:09.672122 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9b7b0c-9921-4f29-b542-9cac403be2d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:09.672917 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9b7b0c-9921-4f29-b542-9cac403be2d9 None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:09.673420 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9b7b0c-9921-4f29-b542-9cac403be2d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:09.673691 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9b7b0c-9921-4f29-b542-9cac403be2d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:09.674782 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9b7b0c-9921-4f29-b542-9cac403be2d9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:09.676714 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9b7b0c-9921-4f29-b542-9cac403be2d9 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:09.738989 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd64eac-363b-4e07-8dfa-b21b48f12d2a None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:09.773800 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af384e62-dc4d-4de0-a04c-956540a53882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:09.774451 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af384e62-dc4d-4de0-a04c-956540a53882 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:09.774727 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af384e62-dc4d-4de0-a04c-956540a53882 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:09.775044 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af384e62-dc4d-4de0-a04c-956540a53882 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:09.775693 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af384e62-dc4d-4de0-a04c-956540a53882 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:09.775990 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af384e62-dc4d-4de0-a04c-956540a53882 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:09.777445 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af384e62-dc4d-4de0-a04c-956540a53882 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:09.797052 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:09.797354 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 19/33] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:02:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:09.799160 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af384e62-dc4d-4de0-a04c-956540a53882 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:09.806984 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 15/34] 10.222.0.43 () {60 vars in 1131 bytes} [Tue Sep 6 16:02:09 2022] POST /identity/v3/roles => generated 219 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:02:09.808588 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb11fc-dae2-4413-9479-7456aa8f023f None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:09.808983 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb11fc-dae2-4413-9479-7456aa8f023f None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:09.809333 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb11fc-dae2-4413-9479-7456aa8f023f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:09.841788 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-adeb11fc-dae2-4413-9479-7456aa8f023f None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:09.846925 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-adeb11fc-dae2-4413-9479-7456aa8f023f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:09.884513 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:09.966569 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af384e62-dc4d-4de0-a04c-956540a53882 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:10.001262 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 16/35] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:02:10.002143 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeb11fc-dae2-4413-9479-7456aa8f023f None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:10.014542 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c1306b-0720-428a-a053-53a8f55cde57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:10.015194 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c1306b-0720-428a-a053-53a8f55cde57 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:10.015468 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c1306b-0720-428a-a053-53a8f55cde57 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:10.015737 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c1306b-0720-428a-a053-53a8f55cde57 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:10.021449 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c1306b-0720-428a-a053-53a8f55cde57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:10.022944 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c1306b-0720-428a-a053-53a8f55cde57 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:10.024620 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 20/36] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:02:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:10.044140 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c1306b-0720-428a-a053-53a8f55cde57 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:10.114048 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeb11fc-dae2-4413-9479-7456aa8f023f None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:10.125775 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:10.152102 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67970fe6-4ba1-4f56-b4b1-3d8cdd66700a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:10.152944 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67970fe6-4ba1-4f56-b4b1-3d8cdd66700a None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:10.153225 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67970fe6-4ba1-4f56-b4b1-3d8cdd66700a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:10.153779 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67970fe6-4ba1-4f56-b4b1-3d8cdd66700a None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:10.154696 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67970fe6-4ba1-4f56-b4b1-3d8cdd66700a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:10.156704 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67970fe6-4ba1-4f56-b4b1-3d8cdd66700a None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:10.169854 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c1306b-0720-428a-a053-53a8f55cde57 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:10.190627 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:10.201850 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 17/37] 10.222.0.43 () {60 vars in 1141 bytes} [Tue Sep 6 16:02:10 2022] POST /identity/v3/projects => generated 301 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:02:10.203520 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94e13d3-31d4-4e14-a892-82445eff35b6 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:10.204074 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94e13d3-31d4-4e14-a892-82445eff35b6 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:10.204348 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94e13d3-31d4-4e14-a892-82445eff35b6 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:10.206475 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 18/38] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:02:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:02:10.213249 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb36481-df9f-4453-875f-5487016eff36 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:10.213546 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb36481-df9f-4453-875f-5487016eff36 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:10.213773 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb36481-df9f-4453-875f-5487016eff36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:10.225062 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa19bc5-1eb2-4b02-9986-fd0bac236daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:10.225821 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa19bc5-1eb2-4b02-9986-fd0bac236daa None admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:10.226167 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa19bc5-1eb2-4b02-9986-fd0bac236daa None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:10.226545 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa19bc5-1eb2-4b02-9986-fd0bac236daa None admin] PATH_INFO: `/v3/domains/default/users/824c9880f41b4962b83243abf75b8162/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:10.228789 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa19bc5-1eb2-4b02-9986-fd0bac236daa None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=824c9880f41b4962b83243abf75b8162, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:10.230569 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa19bc5-1eb2-4b02-9986-fd0bac236daa None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:10.247861 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-adb36481-df9f-4453-875f-5487016eff36 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:10.248930 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-adb36481-df9f-4453-875f-5487016eff36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:10.352858 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 19/39] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:02:10.354642 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f13bd0-5b18-4d41-96d9-4c61edc78934 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:10.355156 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f13bd0-5b18-4d41-96d9-4c61edc78934 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:10.355721 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f13bd0-5b18-4d41-96d9-4c61edc78934 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:10.357337 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 20/40] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:02:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:02:10.362697 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d92a5-3749-422d-9375-fa53ab176175 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:10.363075 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 21/41] 10.222.0.43 () {58 vars in 1361 bytes} [Tue Sep 6 16:02:10 2022] PUT /identity/v3/domains/default/users/824c9880f41b4962b83243abf75b8162/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:02:10.363463 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d92a5-3749-422d-9375-fa53ab176175 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:10.363973 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d92a5-3749-422d-9375-fa53ab176175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:10.385769 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-094d92a5-3749-422d-9375-fa53ab176175 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:10.387361 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-094d92a5-3749-422d-9375-fa53ab176175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:10.403330 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa19bc5-1eb2-4b02-9986-fd0bac236daa None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:10.480712 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 21/42] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 16:02:10.542928 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa19bc5-1eb2-4b02-9986-fd0bac236daa None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:10.566457 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:10.578747 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f25f5f4-58d4-4c07-ae50-09dca058302d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:10.579369 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f25f5f4-58d4-4c07-ae50-09dca058302d None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:10.579690 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f25f5f4-58d4-4c07-ae50-09dca058302d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:10.580288 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f25f5f4-58d4-4c07-ae50-09dca058302d None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:10.581389 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f25f5f4-58d4-4c07-ae50-09dca058302d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:10.583796 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f25f5f4-58d4-4c07-ae50-09dca058302d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:10.604989 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 22/43] 10.222.0.43 () {60 vars in 1131 bytes} [Tue Sep 6 16:02:10 2022] POST /identity/v3/roles => generated 225 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:02:10.663554 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094d92a5-3749-422d-9375-fa53ab176175 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:10.732570 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:10.764557 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f25f5f4-58d4-4c07-ae50-09dca058302d None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:10.784175 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094d92a5-3749-422d-9375-fa53ab176175 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:10.827646 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417d3e5b-35ac-4776-bfcc-1b3da4a8694c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:10.828178 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417d3e5b-35ac-4776-bfcc-1b3da4a8694c None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:10.828451 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417d3e5b-35ac-4776-bfcc-1b3da4a8694c None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:10.828926 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417d3e5b-35ac-4776-bfcc-1b3da4a8694c None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:10.829800 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417d3e5b-35ac-4776-bfcc-1b3da4a8694c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:10.831397 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417d3e5b-35ac-4776-bfcc-1b3da4a8694c None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:10.853659 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 22/44] 10.222.0.43 () {60 vars in 1131 bytes} [Tue Sep 6 16:02:10 2022] POST /identity/v3/roles => generated 223 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:02:11.009300 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f25f5f4-58d4-4c07-ae50-09dca058302d None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:11.043100 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ec22da-00d3-40d5-8706-5e7a7b3baaa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:11.043739 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec22da-00d3-40d5-8706-5e7a7b3baaa5 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:11.044019 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec22da-00d3-40d5-8706-5e7a7b3baaa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:11.044293 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec22da-00d3-40d5-8706-5e7a7b3baaa5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:11.045530 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ec22da-00d3-40d5-8706-5e7a7b3baaa5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:11.046996 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ec22da-00d3-40d5-8706-5e7a7b3baaa5 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:11.048571 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 23/45] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:02:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:02:11.054171 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417d3e5b-35ac-4776-bfcc-1b3da4a8694c None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:11.153735 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417d3e5b-35ac-4776-bfcc-1b3da4a8694c None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:11.173432 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:11.189034 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfa6548-6e7c-4317-a3ca-be61b99e827c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:11.189642 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfa6548-6e7c-4317-a3ca-be61b99e827c None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:11.189918 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfa6548-6e7c-4317-a3ca-be61b99e827c None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:11.190194 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfa6548-6e7c-4317-a3ca-be61b99e827c None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:11.190830 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfa6548-6e7c-4317-a3ca-be61b99e827c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:11.192292 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfa6548-6e7c-4317-a3ca-be61b99e827c None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:11.218873 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 23/46] 10.222.0.43 () {60 vars in 1141 bytes} [Tue Sep 6 16:02:11 2022] POST /identity/v3/projects => generated 312 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:02:12.358951 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bf31f1-56c5-4e40-b4d2-b88f78ed97f1 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:12.359465 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bf31f1-56c5-4e40-b4d2-b88f78ed97f1 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:12.359465 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bf31f1-56c5-4e40-b4d2-b88f78ed97f1 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:12.360901 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 24/47] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:02:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:02:12.366116 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eb0d29-d0f6-49ee-b1cc-68e521f53b50 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:12.366524 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eb0d29-d0f6-49ee-b1cc-68e521f53b50 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:12.366652 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eb0d29-d0f6-49ee-b1cc-68e521f53b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:12.386921 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-03eb0d29-d0f6-49ee-b1cc-68e521f53b50 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:12.388833 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-03eb0d29-d0f6-49ee-b1cc-68e521f53b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:12.457612 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 24/48] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:02:12.722953 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bf31f1-56c5-4e40-b4d2-b88f78ed97f1 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:12.781367 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:12.837977 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bf31f1-56c5-4e40-b4d2-b88f78ed97f1 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:12.872934 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b104624-7046-459c-8990-d4f25587ed68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:12.873561 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b104624-7046-459c-8990-d4f25587ed68 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:12.873841 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b104624-7046-459c-8990-d4f25587ed68 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:12.874113 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b104624-7046-459c-8990-d4f25587ed68 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:12.884408 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b104624-7046-459c-8990-d4f25587ed68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:02:12.889354 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-9b104624-7046-459c-8990-d4f25587ed68 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:02:12.889354 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:12.889354 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:12.889354 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:12.889354 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:12.889354 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:12.889354 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:12.889354 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:12.889354 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:12.889354 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:12.889354 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:12.889354 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:12.889354 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:02:12.889354 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:02:12.889354 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:02:12.889354 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:02:12.890197 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:02:12.890197 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:02:12.890197 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:02:12.890197 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:02:12.890197 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:12.890197 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:12.890197 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:12.890197 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:12.890197 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:12.890197 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:12.890197 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:12.890197 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:12.890197 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:12.890197 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:12.890197 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:12.890197 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:12.891179 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:12.891179 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:12.891179 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:12.891179 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:12.891179 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:02:12.891179 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:02:12.891179 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:02:12.891179 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:02:12.891179 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:02:12.891179 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:02:12.891179 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:02:12.891179 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:02:12.891179 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 25/49] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 6 16:02:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:02:12.896435 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eb0d29-d0f6-49ee-b1cc-68e521f53b50 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:13.000948 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eb0d29-d0f6-49ee-b1cc-68e521f53b50 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:13.002423 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:13.087750 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0895f20-7690-45fc-9125-9e3be0d9cace None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:13.088022 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0895f20-7690-45fc-9125-9e3be0d9cace None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:13.088294 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0895f20-7690-45fc-9125-9e3be0d9cace None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:13.089695 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 26/50] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:02:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:02:13.095528 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac45261-3f4f-42ec-82bf-6a4e144b1bc1 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:13.095814 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac45261-3f4f-42ec-82bf-6a4e144b1bc1 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:13.096089 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac45261-3f4f-42ec-82bf-6a4e144b1bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:13.116158 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-cac45261-3f4f-42ec-82bf-6a4e144b1bc1 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:13.118291 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-cac45261-3f4f-42ec-82bf-6a4e144b1bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:13.149217 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b926222-79db-4d0f-a239-e0e672e1274d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:13.149833 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b926222-79db-4d0f-a239-e0e672e1274d None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:13.150119 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b926222-79db-4d0f-a239-e0e672e1274d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:13.150467 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b926222-79db-4d0f-a239-e0e672e1274d None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:13.151334 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b926222-79db-4d0f-a239-e0e672e1274d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:13.151462 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b926222-79db-4d0f-a239-e0e672e1274d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:13.152807 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b926222-79db-4d0f-a239-e0e672e1274d None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:13.164604 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 25/51] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 6 16:02:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:13.170289 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b926222-79db-4d0f-a239-e0e672e1274d None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:13.188337 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 27/52] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:02:13.272306 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b926222-79db-4d0f-a239-e0e672e1274d None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:13.306573 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05843af9-a87a-4145-aea9-f4b48e502952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:13.307214 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05843af9-a87a-4145-aea9-f4b48e502952 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:13.307490 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05843af9-a87a-4145-aea9-f4b48e502952 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:13.307756 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05843af9-a87a-4145-aea9-f4b48e502952 None admin] PATH_INFO: `/v3/users/824c9880f41b4962b83243abf75b8162` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:13.309299 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05843af9-a87a-4145-aea9-f4b48e502952 None admin] RBAC: Authorizing `identity:get_user(user_id=824c9880f41b4962b83243abf75b8162)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:13.310765 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05843af9-a87a-4145-aea9-f4b48e502952 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:13.312367 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 26/53] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 6 16:02:13 2022] GET /identity/v3/users/824c9880f41b4962b83243abf75b8162 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:02:13.318221 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac45261-3f4f-42ec-82bf-6a4e144b1bc1 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:13.359780 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:13.418195 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac45261-3f4f-42ec-82bf-6a4e144b1bc1 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:13.453061 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9019a2e-2d9b-4e4a-b8d6-4d5aeafac26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:13.453698 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9019a2e-2d9b-4e4a-b8d6-4d5aeafac26d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:13.453976 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9019a2e-2d9b-4e4a-b8d6-4d5aeafac26d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:13.454248 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9019a2e-2d9b-4e4a-b8d6-4d5aeafac26d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:13.455369 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9019a2e-2d9b-4e4a-b8d6-4d5aeafac26d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:13.456872 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9019a2e-2d9b-4e4a-b8d6-4d5aeafac26d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:13.458093 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05843af9-a87a-4145-aea9-f4b48e502952 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:13.458799 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 28/54] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:02:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:13.464425 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9019a2e-2d9b-4e4a-b8d6-4d5aeafac26d None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:13.489465 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:13.559709 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05843af9-a87a-4145-aea9-f4b48e502952 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:13.560410 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9019a2e-2d9b-4e4a-b8d6-4d5aeafac26d None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:13.570816 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:13.593925 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361ba07c-4fef-4ede-b046-0ab74fc6cfb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:13.594562 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ba07c-4fef-4ede-b046-0ab74fc6cfb4 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:13.594840 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ba07c-4fef-4ede-b046-0ab74fc6cfb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:13.595111 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ba07c-4fef-4ede-b046-0ab74fc6cfb4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:13.596194 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361ba07c-4fef-4ede-b046-0ab74fc6cfb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:13.597013 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320787bd-dffe-4ced-b877-6bf9cf1623b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:13.597626 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320787bd-dffe-4ced-b877-6bf9cf1623b5 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:13.597715 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361ba07c-4fef-4ede-b046-0ab74fc6cfb4 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:13.597909 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320787bd-dffe-4ced-b877-6bf9cf1623b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:13.598183 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320787bd-dffe-4ced-b877-6bf9cf1623b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:13.598906 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320787bd-dffe-4ced-b877-6bf9cf1623b5 None admin] RBAC: Adding query filter params (role.id=edaabdea8be84a7980fb2c651fe52f63, scope.domain.id=default, user.id=824c9880f41b4962b83243abf75b8162) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:13.599203 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320787bd-dffe-4ced-b877-6bf9cf1623b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:13.599323 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 27/55] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:02:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:13.600724 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320787bd-dffe-4ced-b877-6bf9cf1623b5 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:13.604806 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361ba07c-4fef-4ede-b046-0ab74fc6cfb4 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:13.608199 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 29/56] 10.222.0.43 () {56 vars in 1322 bytes} [Tue Sep 6 16:02:13 2022] GET /identity/v3/role_assignments?user.id=824c9880f41b4962b83243abf75b8162&scope.domain.id=default&role.id=edaabdea8be84a7980fb2c651fe52f63 => generated 530 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:02:13.709732 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:13.713354 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361ba07c-4fef-4ede-b046-0ab74fc6cfb4 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:13.748599 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f70f21-0200-4cd4-a719-84e87af2624e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:13.749209 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f70f21-0200-4cd4-a719-84e87af2624e None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:13.749532 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f70f21-0200-4cd4-a719-84e87af2624e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:13.749795 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f70f21-0200-4cd4-a719-84e87af2624e None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:13.750452 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f70f21-0200-4cd4-a719-84e87af2624e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:13.751929 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f70f21-0200-4cd4-a719-84e87af2624e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:13.780050 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 28/57] 10.222.0.43 () {60 vars in 1140 bytes} [Tue Sep 6 16:02:13 2022] POST /identity/v3/projects => generated 298 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:02:15.655293 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad18f79-f9ef-409b-afdf-a5ee86e6a109 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:15.655805 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad18f79-f9ef-409b-afdf-a5ee86e6a109 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:15.655805 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad18f79-f9ef-409b-afdf-a5ee86e6a109 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:15.657148 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 30/58] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:02:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:02:15.663444 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af03fab-33d3-4b85-8270-34fbb0301589 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:15.664182 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af03fab-33d3-4b85-8270-34fbb0301589 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:15.664370 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af03fab-33d3-4b85-8270-34fbb0301589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:15.684093 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-8af03fab-33d3-4b85-8270-34fbb0301589 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:15.685335 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8af03fab-33d3-4b85-8270-34fbb0301589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:15.754404 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 29/59] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:02:16.019803 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad18f79-f9ef-409b-afdf-a5ee86e6a109 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:16.127470 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad18f79-f9ef-409b-afdf-a5ee86e6a109 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:16.162102 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297fd282-3104-46c5-8609-ff3637e5a61a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:16.162713 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297fd282-3104-46c5-8609-ff3637e5a61a None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:16.162988 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297fd282-3104-46c5-8609-ff3637e5a61a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:16.163253 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297fd282-3104-46c5-8609-ff3637e5a61a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:16.164584 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297fd282-3104-46c5-8609-ff3637e5a61a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:16.166074 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297fd282-3104-46c5-8609-ff3637e5a61a None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:16.167607 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 31/60] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:02:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:16.173367 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af03fab-33d3-4b85-8270-34fbb0301589 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:16.255360 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:16.276829 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af03fab-33d3-4b85-8270-34fbb0301589 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:16.312386 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecedcc11-2a77-419a-a87c-34c65a53e6f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:16.313019 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecedcc11-2a77-419a-a87c-34c65a53e6f6 None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:16.313296 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecedcc11-2a77-419a-a87c-34c65a53e6f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:16.313566 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecedcc11-2a77-419a-a87c-34c65a53e6f6 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:16.314207 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecedcc11-2a77-419a-a87c-34c65a53e6f6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:16.315658 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecedcc11-2a77-419a-a87c-34c65a53e6f6 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:16.348555 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 30/61] 10.222.0.43 () {60 vars in 1132 bytes} [Tue Sep 6 16:02:16 2022] POST /identity/v3/users => generated 275 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:02:19.179021 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a35d829-8be1-45fe-a103-7bad5a681e8e None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:19.180072 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a35d829-8be1-45fe-a103-7bad5a681e8e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:19.180544 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a35d829-8be1-45fe-a103-7bad5a681e8e None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:19.182074 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 32/62] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:02:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:02:19.189589 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e879217-eee1-4109-a4de-82ab5ea4e31b None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:19.190336 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e879217-eee1-4109-a4de-82ab5ea4e31b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:19.190747 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e879217-eee1-4109-a4de-82ab5ea4e31b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:19.215637 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-5e879217-eee1-4109-a4de-82ab5ea4e31b None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:19.217157 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5e879217-eee1-4109-a4de-82ab5ea4e31b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:19.309196 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 31/63] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:02:19.475193 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b89ea6-a74a-4858-813b-caf5993fafe6 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:19.475488 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b89ea6-a74a-4858-813b-caf5993fafe6 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:19.475654 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b89ea6-a74a-4858-813b-caf5993fafe6 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:19.477369 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 33/64] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:02:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:02:19.482713 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34c278f-0011-4211-8a12-5de5e35894a3 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:19.483173 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34c278f-0011-4211-8a12-5de5e35894a3 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:19.483591 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34c278f-0011-4211-8a12-5de5e35894a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:19.505840 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-d34c278f-0011-4211-8a12-5de5e35894a3 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:19.507056 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d34c278f-0011-4211-8a12-5de5e35894a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:19.576359 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cd78e3-d057-44ce-ba31-3a52a613a2bc None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:19.576637 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cd78e3-d057-44ce-ba31-3a52a613a2bc None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:19.577165 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cd78e3-d057-44ce-ba31-3a52a613a2bc None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:19.578786 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 34/65] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:02:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:02:19.591716 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:19.596472 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6875d6b7-98ef-476c-8e3a-424b50d0c741 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:19.597034 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6875d6b7-98ef-476c-8e3a-424b50d0c741 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:19.597495 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6875d6b7-98ef-476c-8e3a-424b50d0c741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:19.613072 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 32/66] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:02:19.627248 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-6875d6b7-98ef-476c-8e3a-424b50d0c741 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:19.628722 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6875d6b7-98ef-476c-8e3a-424b50d0c741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:19.677945 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:19.726534 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 35/67] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 16:02:19.733027 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34c278f-0011-4211-8a12-5de5e35894a3 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:19.790590 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78383b1a-999a-4f1f-a2eb-87f7a2fa29a0 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:19.791160 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78383b1a-999a-4f1f-a2eb-87f7a2fa29a0 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:19.791606 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78383b1a-999a-4f1f-a2eb-87f7a2fa29a0 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:19.794867 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 36/68] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:02:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:02:19.798700 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f56243-f10f-4b9b-9ef6-cd1028bae34f None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:19.799185 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f56243-f10f-4b9b-9ef6-cd1028bae34f None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:19.799756 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f56243-f10f-4b9b-9ef6-cd1028bae34f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:19.835146 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-97f56243-f10f-4b9b-9ef6-cd1028bae34f None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:19.837417 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-97f56243-f10f-4b9b-9ef6-cd1028bae34f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:19.889615 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34c278f-0011-4211-8a12-5de5e35894a3 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:19.955454 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529629d4-a7d8-4a5a-9e65-51c9c1f9365f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:19.956102 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529629d4-a7d8-4a5a-9e65-51c9c1f9365f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:19.956378 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529629d4-a7d8-4a5a-9e65-51c9c1f9365f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:19.956650 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529629d4-a7d8-4a5a-9e65-51c9c1f9365f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:19.964139 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 37/69] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 16:02:19.977390 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529629d4-a7d8-4a5a-9e65-51c9c1f9365f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:02:19.982351 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-529629d4-a7d8-4a5a-9e65-51c9c1f9365f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 16:02:19.982351 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:19.982351 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:19.982351 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:19.982351 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:19.982351 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:19.982351 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:19.982351 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:19.982351 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:19.982351 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:19.982351 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:19.982351 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:19.982351 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:02:19.982351 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:02:19.982351 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:02:19.982351 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:02:19.983177 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:02:19.983177 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:02:19.983177 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:02:19.983177 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:02:19.983177 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:19.983177 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:19.983177 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:19.983177 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:19.983177 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:19.983177 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:19.983177 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:19.983177 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:19.983177 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:19.983177 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:19.983177 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:19.983177 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:19.983933 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:19.983933 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:19.983933 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:19.983933 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:19.983933 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:02:19.983933 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:02:19.983933 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:02:19.983933 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:02:19.983933 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:02:19.983933 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:02:19.983933 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 16:02:19.983933 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:02:19.983933 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 33/70] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 6 16:02:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:02:19.990427 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f56243-f10f-4b9b-9ef6-cd1028bae34f None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:20.013625 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529629d4-a7d8-4a5a-9e65-51c9c1f9365f None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:20.095266 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:20.116162 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:20.121283 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f56243-f10f-4b9b-9ef6-cd1028bae34f None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:20.141188 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529629d4-a7d8-4a5a-9e65-51c9c1f9365f None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:20.161359 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d20e675-8ffa-4a8d-b165-76f3ed691bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:20.161985 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d20e675-8ffa-4a8d-b165-76f3ed691bcd None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:20.162263 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d20e675-8ffa-4a8d-b165-76f3ed691bcd None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:20.162782 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d20e675-8ffa-4a8d-b165-76f3ed691bcd None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:20.163235 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d20e675-8ffa-4a8d-b165-76f3ed691bcd None admin] RBAC: Adding query filter params (name=member) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:20.163531 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d20e675-8ffa-4a8d-b165-76f3ed691bcd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:20.165397 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d20e675-8ffa-4a8d-b165-76f3ed691bcd None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:20.178063 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 38/71] 10.222.0.43 () {56 vars in 1101 bytes} [Tue Sep 6 16:02:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:20.178063 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d20e675-8ffa-4a8d-b165-76f3ed691bcd None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:20.182718 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c0247a-3b41-4b5c-a951-737cc225e20e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:20.183365 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c0247a-3b41-4b5c-a951-737cc225e20e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:20.183650 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c0247a-3b41-4b5c-a951-737cc225e20e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:20.183924 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c0247a-3b41-4b5c-a951-737cc225e20e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:20.194469 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c0247a-3b41-4b5c-a951-737cc225e20e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:02:20.199466 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-43c0247a-3b41-4b5c-a951-737cc225e20e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 16:02:20.199466 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:20.199466 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:20.199466 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:20.199466 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:20.199466 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:20.199466 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:20.199466 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:20.199466 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:20.199466 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:20.199466 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:20.199466 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:20.199466 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:02:20.199466 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:02:20.199466 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:02:20.199466 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:02:20.200284 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:02:20.200284 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:02:20.200284 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:02:20.200284 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:02:20.200284 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:20.200284 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:20.200284 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:20.200284 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:20.200284 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:20.200284 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:20.200284 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:20.200284 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:20.200284 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:20.200284 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:20.200284 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:20.200284 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:20.201095 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:20.201095 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:20.201095 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:20.201095 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:20.201095 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:02:20.201095 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:02:20.201095 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:02:20.201095 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:02:20.201095 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:02:20.201095 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:02:20.201095 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 16:02:20.201095 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:02:20.201095 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 34/72] 10.222.0.43 () {56 vars in 1114 bytes} [Tue Sep 6 16:02:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:02:20.201730 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a213e4c7-b73d-4cab-8635-ee6a503136df None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:20.201994 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a213e4c7-b73d-4cab-8635-ee6a503136df None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:20.202266 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a213e4c7-b73d-4cab-8635-ee6a503136df None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:20.204286 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 35/73] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:02:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:02:20.205044 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a213e4c7-b73d-4cab-8635-ee6a503136df None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:20.289544 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d20e675-8ffa-4a8d-b165-76f3ed691bcd None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:20.322523 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ea930e-8f52-4c12-a100-68170b3ec7fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:20.322731 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a213e4c7-b73d-4cab-8635-ee6a503136df None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:20.323158 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ea930e-8f52-4c12-a100-68170b3ec7fe None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:20.323444 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ea930e-8f52-4c12-a100-68170b3ec7fe None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:20.323720 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ea930e-8f52-4c12-a100-68170b3ec7fe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:20.334344 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ea930e-8f52-4c12-a100-68170b3ec7fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:02:20.339728 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-18ea930e-8f52-4c12-a100-68170b3ec7fe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:02:20.339728 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:20.339728 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:20.339728 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:20.339728 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:20.339728 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:20.339728 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:20.339728 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:20.339728 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:20.339728 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:20.339728 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:20.339728 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:20.339728 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:02:20.339728 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:02:20.339728 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:02:20.339728 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:02:20.340772 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:02:20.340772 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:02:20.340772 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:02:20.340772 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:02:20.340772 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:20.340772 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:20.340772 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:20.340772 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:20.340772 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:20.340772 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:20.340772 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:20.340772 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:20.340772 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:20.340772 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:20.340772 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:20.340772 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:20.341747 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:20.341747 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:20.341747 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:20.341747 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:20.341747 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:02:20.341747 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:02:20.341747 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:02:20.341747 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:02:20.341747 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:02:20.341747 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:02:20.341747 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:02:20.341747 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:02:20.342519 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:20.342585 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 39/74] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 6 16:02:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:02:20.343801 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ea930e-8f52-4c12-a100-68170b3ec7fe None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:20.358610 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115b85a7-380f-4bbc-a6b8-851ea95e9e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:20.359250 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115b85a7-380f-4bbc-a6b8-851ea95e9e1e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:20.359505 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115b85a7-380f-4bbc-a6b8-851ea95e9e1e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:20.359782 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115b85a7-380f-4bbc-a6b8-851ea95e9e1e None admin] PATH_INFO: `/v3/users/6962122ffdbe4d6288f017e98d1e8ecb` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:20.369429 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:20.374861 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115b85a7-380f-4bbc-a6b8-851ea95e9e1e None admin] RBAC: Authorizing `identity:get_user(user_id=6962122ffdbe4d6288f017e98d1e8ecb)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:20.376333 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115b85a7-380f-4bbc-a6b8-851ea95e9e1e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:20.378009 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 36/75] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 6 16:02:20 2022] GET /identity/v3/users/6962122ffdbe4d6288f017e98d1e8ecb => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:20.380899 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ec3e8f-7428-458a-acee-dae711b1f561 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:20.380899 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ec3e8f-7428-458a-acee-dae711b1f561 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:20.380899 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ec3e8f-7428-458a-acee-dae711b1f561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:20.397164 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-07ec3e8f-7428-458a-acee-dae711b1f561 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:20.398349 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-07ec3e8f-7428-458a-acee-dae711b1f561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:20.447435 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ea930e-8f52-4c12-a100-68170b3ec7fe None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:20.464889 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 37/76] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:02:20.464889 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ec3e8f-7428-458a-acee-dae711b1f561 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:20.482210 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a620cb3c-139c-4319-85ec-993fcb72c070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:20.482829 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a620cb3c-139c-4319-85ec-993fcb72c070 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:20.483109 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a620cb3c-139c-4319-85ec-993fcb72c070 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:20.483379 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a620cb3c-139c-4319-85ec-993fcb72c070 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:20.484048 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a620cb3c-139c-4319-85ec-993fcb72c070 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:20.484348 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a620cb3c-139c-4319-85ec-993fcb72c070 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:20.485796 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a620cb3c-139c-4319-85ec-993fcb72c070 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:20.494754 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 40/77] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 6 16:02:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:20.495383 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a620cb3c-139c-4319-85ec-993fcb72c070 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:20.571207 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ec3e8f-7428-458a-acee-dae711b1f561 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:20.579665 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:20.587789 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:20.604437 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a620cb3c-139c-4319-85ec-993fcb72c070 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:20.607656 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e12eeec-5cb2-45c2-a447-dcd97c4162e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:20.608285 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e12eeec-5cb2-45c2-a447-dcd97c4162e1 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:20.608563 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e12eeec-5cb2-45c2-a447-dcd97c4162e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:20.608859 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e12eeec-5cb2-45c2-a447-dcd97c4162e1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:20.609971 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e12eeec-5cb2-45c2-a447-dcd97c4162e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:20.611425 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e12eeec-5cb2-45c2-a447-dcd97c4162e1 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:20.613220 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 38/78] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:02:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:20.613782 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e12eeec-5cb2-45c2-a447-dcd97c4162e1 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:20.644424 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba704f6-4cf3-4215-a0ea-e47779ca4f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:20.645072 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba704f6-4cf3-4215-a0ea-e47779ca4f82 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:20.645354 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba704f6-4cf3-4215-a0ea-e47779ca4f82 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:20.645627 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba704f6-4cf3-4215-a0ea-e47779ca4f82 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:20.646300 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba704f6-4cf3-4215-a0ea-e47779ca4f82 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:20.646602 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba704f6-4cf3-4215-a0ea-e47779ca4f82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:20.648620 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba704f6-4cf3-4215-a0ea-e47779ca4f82 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:20.656482 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 41/79] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 6 16:02:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:20.657523 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba704f6-4cf3-4215-a0ea-e47779ca4f82 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:20.717094 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e12eeec-5cb2-45c2-a447-dcd97c4162e1 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:20.751383 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0dc388-5532-46c3-85ff-cad5de1af959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:20.751992 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0dc388-5532-46c3-85ff-cad5de1af959 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:20.752290 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0dc388-5532-46c3-85ff-cad5de1af959 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:20.752543 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0dc388-5532-46c3-85ff-cad5de1af959 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:20.763917 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba704f6-4cf3-4215-a0ea-e47779ca4f82 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:20.793248 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:20.799226 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0d442f-119a-4b85-897c-cea76f8656d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:20.799869 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0d442f-119a-4b85-897c-cea76f8656d5 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:20.800146 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0d442f-119a-4b85-897c-cea76f8656d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:20.800439 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0d442f-119a-4b85-897c-cea76f8656d5 None admin] PATH_INFO: `/v3/users/6962122ffdbe4d6288f017e98d1e8ecb` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:20.801676 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0d442f-119a-4b85-897c-cea76f8656d5 None admin] RBAC: Authorizing `identity:get_user(user_id=6962122ffdbe4d6288f017e98d1e8ecb)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:20.803146 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0d442f-119a-4b85-897c-cea76f8656d5 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:20.804814 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 42/80] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 6 16:02:20 2022] GET /identity/v3/users/6962122ffdbe4d6288f017e98d1e8ecb => generated 275 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:20.805563 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0d442f-119a-4b85-897c-cea76f8656d5 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:20.820920 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:20.829348 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0dc388-5532-46c3-85ff-cad5de1af959 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:20.830766 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0dc388-5532-46c3-85ff-cad5de1af959 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:20.909939 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0d442f-119a-4b85-897c-cea76f8656d5 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:20.928608 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 39/81] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 6 16:02:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 16:02:20.929469 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0dc388-5532-46c3-85ff-cad5de1af959 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:20.947817 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a36832-cf8f-4e42-85c7-411f68500975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:20.948436 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a36832-cf8f-4e42-85c7-411f68500975 None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:20.948708 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a36832-cf8f-4e42-85c7-411f68500975 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:20.949015 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a36832-cf8f-4e42-85c7-411f68500975 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:20.949655 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a36832-cf8f-4e42-85c7-411f68500975 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:20.951114 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a36832-cf8f-4e42-85c7-411f68500975 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:20.978329 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 43/82] 10.222.0.43 () {60 vars in 1132 bytes} [Tue Sep 6 16:02:20 2022] POST /identity/v3/users => generated 289 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:02:20.979143 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a36832-cf8f-4e42-85c7-411f68500975 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:21.007032 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:21.035468 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0dc388-5532-46c3-85ff-cad5de1af959 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:21.060022 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:21.072629 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb66187-2e11-436b-b6d1-2721b20efef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:21.073316 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb66187-2e11-436b-b6d1-2721b20efef3 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:21.073595 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb66187-2e11-436b-b6d1-2721b20efef3 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:21.073866 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb66187-2e11-436b-b6d1-2721b20efef3 None admin] PATH_INFO: `/v3/users/824c9880f41b4962b83243abf75b8162` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:21.075042 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb66187-2e11-436b-b6d1-2721b20efef3 None admin] RBAC: Authorizing `identity:get_user(user_id=824c9880f41b4962b83243abf75b8162)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:21.076525 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb66187-2e11-436b-b6d1-2721b20efef3 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:21.078199 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 40/83] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 6 16:02:20 2022] GET /identity/v3/users/824c9880f41b4962b83243abf75b8162 => generated 247 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:21.078991 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb66187-2e11-436b-b6d1-2721b20efef3 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:21.101103 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a36832-cf8f-4e42-85c7-411f68500975 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:21.140276 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba4e236-3a86-496a-a6ff-bb62b6e1dae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:21.140653 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba4e236-3a86-496a-a6ff-bb62b6e1dae5 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:21.140955 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba4e236-3a86-496a-a6ff-bb62b6e1dae5 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:21.141225 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba4e236-3a86-496a-a6ff-bb62b6e1dae5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:21.155123 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba4e236-3a86-496a-a6ff-bb62b6e1dae5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:02:21.160112 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-5ba4e236-3a86-496a-a6ff-bb62b6e1dae5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 16:02:21.160112 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:21.160112 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:21.160112 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:21.160112 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:21.160112 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:21.160112 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:21.160112 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:21.160112 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:21.160112 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:21.160112 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:21.160112 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:21.160112 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:02:21.160112 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:02:21.160112 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:02:21.160112 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:02:21.160998 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:02:21.160998 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:02:21.160998 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:02:21.160998 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:02:21.160998 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:21.160998 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:21.160998 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:21.160998 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:21.160998 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:21.160998 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:21.160998 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:21.160998 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:21.160998 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:21.160998 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:21.160998 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:21.160998 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:21.161751 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:21.161751 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:21.161751 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:21.161751 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:21.161751 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:02:21.161751 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:02:21.161751 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:02:21.161751 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:02:21.161751 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:02:21.161751 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:02:21.161751 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 16:02:21.161751 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:02:21.161751 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 44/84] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 6 16:02:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:02:21.162377 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba4e236-3a86-496a-a6ff-bb62b6e1dae5 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:21.205840 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb66187-2e11-436b-b6d1-2721b20efef3 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:21.257042 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3384104d-05ef-48e2-9153-dc14793a84d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:21.258053 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3384104d-05ef-48e2-9153-dc14793a84d1 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:21.258519 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3384104d-05ef-48e2-9153-dc14793a84d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:21.258971 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3384104d-05ef-48e2-9153-dc14793a84d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:21.275676 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:21.290076 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba4e236-3a86-496a-a6ff-bb62b6e1dae5 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:21.329140 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:21.337234 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3384104d-05ef-48e2-9153-dc14793a84d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:21.337853 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441cecaf-5c34-4295-960e-6a3c2518c4eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:21.338619 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441cecaf-5c34-4295-960e-6a3c2518c4eb None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:21.338765 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3384104d-05ef-48e2-9153-dc14793a84d1 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:21.339017 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441cecaf-5c34-4295-960e-6a3c2518c4eb None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:21.339345 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441cecaf-5c34-4295-960e-6a3c2518c4eb None admin] PATH_INFO: `/v3/projects/5e7a87d573a8446bbcbf9e28149a392f` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:21.340662 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441cecaf-5c34-4295-960e-6a3c2518c4eb None admin] RBAC: Authorizing `identity:get_project(project_id=5e7a87d573a8446bbcbf9e28149a392f)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:21.343072 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441cecaf-5c34-4295-960e-6a3c2518c4eb None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:21.348911 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 45/85] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 6 16:02:21 2022] GET /identity/v3/projects/5e7a87d573a8446bbcbf9e28149a392f => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:21.349783 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441cecaf-5c34-4295-960e-6a3c2518c4eb None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:21.445200 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 41/86] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 6 16:02:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 16:02:21.446225 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3384104d-05ef-48e2-9153-dc14793a84d1 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:21.490954 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:21.493024 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441cecaf-5c34-4295-960e-6a3c2518c4eb None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:21.542945 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683e4d10-6238-4841-8fd6-d717d1becbfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:21.543729 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683e4d10-6238-4841-8fd6-d717d1becbfe None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:21.544103 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683e4d10-6238-4841-8fd6-d717d1becbfe None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:21.544518 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683e4d10-6238-4841-8fd6-d717d1becbfe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:21.557909 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3384104d-05ef-48e2-9153-dc14793a84d1 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:21.593780 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55d7882-9f20-41b1-9020-f6617f9b1d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:21.594400 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d7882-9f20-41b1-9020-f6617f9b1d7e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:21.594677 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d7882-9f20-41b1-9020-f6617f9b1d7e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:21.594948 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d7882-9f20-41b1-9020-f6617f9b1d7e None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:21.595622 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55d7882-9f20-41b1-9020-f6617f9b1d7e None admin] RBAC: Adding query filter params (name=member) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:21.595925 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55d7882-9f20-41b1-9020-f6617f9b1d7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:21.597417 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55d7882-9f20-41b1-9020-f6617f9b1d7e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:21.606684 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 42/87] 10.222.0.43 () {56 vars in 1101 bytes} [Tue Sep 6 16:02:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:21.611916 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55d7882-9f20-41b1-9020-f6617f9b1d7e None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:21.625657 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:21.630630 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683e4d10-6238-4841-8fd6-d717d1becbfe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:21.632138 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683e4d10-6238-4841-8fd6-d717d1becbfe None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:21.736464 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 46/88] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 6 16:02:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 16:02:21.737197 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683e4d10-6238-4841-8fd6-d717d1becbfe None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:21.741413 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55d7882-9f20-41b1-9020-f6617f9b1d7e None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:21.744090 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:21.783548 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8d3ad9-681c-4884-9c8c-434d36572928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:21.784424 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d3ad9-681c-4884-9c8c-434d36572928 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:21.784867 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d3ad9-681c-4884-9c8c-434d36572928 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:21.785640 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d3ad9-681c-4884-9c8c-434d36572928 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:21.786899 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8d3ad9-681c-4884-9c8c-434d36572928 None admin] RBAC: Adding query filter params (role.id=15252b0b89894c1e8a6bd9a98b1c3ab7, scope.project.id=5e7a87d573a8446bbcbf9e28149a392f, user.id=6962122ffdbe4d6288f017e98d1e8ecb) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:21.787507 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8d3ad9-681c-4884-9c8c-434d36572928 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:21.789483 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8d3ad9-681c-4884-9c8c-434d36572928 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:21.799986 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 43/89] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 6 16:02:21 2022] GET /identity/v3/role_assignments?user.id=6962122ffdbe4d6288f017e98d1e8ecb&scope.project.id=5e7a87d573a8446bbcbf9e28149a392f&role.id=15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:21.809444 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8d3ad9-681c-4884-9c8c-434d36572928 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:21.897344 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:21.903564 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683e4d10-6238-4841-8fd6-d717d1becbfe None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:21.981765 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e87f6d-f5bc-4dbf-9758-9a590a17b953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:21.982431 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e87f6d-f5bc-4dbf-9758-9a590a17b953 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:21.982708 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e87f6d-f5bc-4dbf-9758-9a590a17b953 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:21.982983 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e87f6d-f5bc-4dbf-9758-9a590a17b953 None admin] PATH_INFO: `/v3/projects/5e7a87d573a8446bbcbf9e28149a392f` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:21.985055 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e87f6d-f5bc-4dbf-9758-9a590a17b953 None admin] RBAC: Authorizing `identity:get_project(project_id=5e7a87d573a8446bbcbf9e28149a392f)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:21.986613 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e87f6d-f5bc-4dbf-9758-9a590a17b953 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:21.988392 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 47/90] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 6 16:02:21 2022] GET /identity/v3/projects/5e7a87d573a8446bbcbf9e28149a392f => generated 298 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:21.991237 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e87f6d-f5bc-4dbf-9758-9a590a17b953 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:22.002276 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8d3ad9-681c-4884-9c8c-434d36572928 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:22.041794 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5467a4e-efea-4ca4-b7d3-ff84897d883a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:22.042470 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5467a4e-efea-4ca4-b7d3-ff84897d883a None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:22.042985 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5467a4e-efea-4ca4-b7d3-ff84897d883a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:22.043154 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5467a4e-efea-4ca4-b7d3-ff84897d883a None admin] PATH_INFO: `/v3/users/6962122ffdbe4d6288f017e98d1e8ecb` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:22.044314 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5467a4e-efea-4ca4-b7d3-ff84897d883a None admin] RBAC: Authorizing `identity:get_user(user_id=6962122ffdbe4d6288f017e98d1e8ecb)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:22.045959 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5467a4e-efea-4ca4-b7d3-ff84897d883a None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:22.048054 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 44/91] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 6 16:02:21 2022] GET /identity/v3/users/6962122ffdbe4d6288f017e98d1e8ecb => generated 275 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:22.052785 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5467a4e-efea-4ca4-b7d3-ff84897d883a None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:22.104467 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:22.140350 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e87f6d-f5bc-4dbf-9758-9a590a17b953 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:22.198438 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d05579f-b86a-4493-8a0d-273eafcf5212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:22.199098 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d05579f-b86a-4493-8a0d-273eafcf5212 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:22.199369 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d05579f-b86a-4493-8a0d-273eafcf5212 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:22.199650 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d05579f-b86a-4493-8a0d-273eafcf5212 None admin] PATH_INFO: `/v3/projects/5e7a87d573a8446bbcbf9e28149a392f` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:22.201411 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d05579f-b86a-4493-8a0d-273eafcf5212 None admin] RBAC: Authorizing `identity:get_project(project_id=5e7a87d573a8446bbcbf9e28149a392f)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:22.202906 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d05579f-b86a-4493-8a0d-273eafcf5212 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:22.207864 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 48/92] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 6 16:02:21 2022] GET /identity/v3/projects/5e7a87d573a8446bbcbf9e28149a392f => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:22.209373 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d05579f-b86a-4493-8a0d-273eafcf5212 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:22.212176 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5467a4e-efea-4ca4-b7d3-ff84897d883a None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:22.280785 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:22.312541 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0a2643-9025-4ab5-a54c-5f9e5b3b5987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:22.313204 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0a2643-9025-4ab5-a54c-5f9e5b3b5987 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:22.313487 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0a2643-9025-4ab5-a54c-5f9e5b3b5987 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:22.313777 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0a2643-9025-4ab5-a54c-5f9e5b3b5987 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:22.314573 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0a2643-9025-4ab5-a54c-5f9e5b3b5987 None admin] RBAC: Adding query filter params (role.id=61df19b017494bdcadc9a5e9d8bb9e52, scope.project.id=5e7a87d573a8446bbcbf9e28149a392f, user.id=6962122ffdbe4d6288f017e98d1e8ecb) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:22.314887 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0a2643-9025-4ab5-a54c-5f9e5b3b5987 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:22.316477 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0a2643-9025-4ab5-a54c-5f9e5b3b5987 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:22.324049 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 45/93] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 6 16:02:22 2022] GET /identity/v3/role_assignments?user.id=6962122ffdbe4d6288f017e98d1e8ecb&scope.project.id=5e7a87d573a8446bbcbf9e28149a392f&role.id=61df19b017494bdcadc9a5e9d8bb9e52 => generated 259 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:22.324049 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0a2643-9025-4ab5-a54c-5f9e5b3b5987 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:22.357926 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d05579f-b86a-4493-8a0d-273eafcf5212 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:22.403768 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d588aa46-4c19-47cf-a006-286e85d2d174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:22.404398 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d588aa46-4c19-47cf-a006-286e85d2d174 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:22.404673 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d588aa46-4c19-47cf-a006-286e85d2d174 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:22.404969 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d588aa46-4c19-47cf-a006-286e85d2d174 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:22.507173 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d588aa46-4c19-47cf-a006-286e85d2d174 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:22.508772 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d588aa46-4c19-47cf-a006-286e85d2d174 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:22.513814 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:22.518080 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0a2643-9025-4ab5-a54c-5f9e5b3b5987 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:22.546890 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:22.582782 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b773ff02-780e-41c3-99fd-e1c8bd89543f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:22.583409 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b773ff02-780e-41c3-99fd-e1c8bd89543f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:22.583858 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b773ff02-780e-41c3-99fd-e1c8bd89543f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:22.584339 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b773ff02-780e-41c3-99fd-e1c8bd89543f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:22.585734 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b773ff02-780e-41c3-99fd-e1c8bd89543f None admin] RBAC: Adding query filter params (role.id=edaabdea8be84a7980fb2c651fe52f63, scope.project.id=5e7a87d573a8446bbcbf9e28149a392f, user.id=824c9880f41b4962b83243abf75b8162) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:22.586362 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b773ff02-780e-41c3-99fd-e1c8bd89543f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:22.587857 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b773ff02-780e-41c3-99fd-e1c8bd89543f None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:22.602720 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 46/94] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 6 16:02:22 2022] GET /identity/v3/role_assignments?user.id=824c9880f41b4962b83243abf75b8162&scope.project.id=5e7a87d573a8446bbcbf9e28149a392f&role.id=edaabdea8be84a7980fb2c651fe52f63 => generated 259 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:22.641435 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 49/95] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 6 16:02:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 16:02:22.647431 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b773ff02-780e-41c3-99fd-e1c8bd89543f None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:22.789999 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b773ff02-780e-41c3-99fd-e1c8bd89543f None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:22.838271 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444d8a5f-128f-44fd-b942-556ce02539de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:22.839367 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d8a5f-128f-44fd-b942-556ce02539de None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:22.840351 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d8a5f-128f-44fd-b942-556ce02539de None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:22.840351 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d8a5f-128f-44fd-b942-556ce02539de None admin] PATH_INFO: `/v3/projects/cfd6384b3080463397c5b619b3626b70` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:22.841191 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444d8a5f-128f-44fd-b942-556ce02539de None admin] RBAC: Authorizing `identity:get_project(project_id=cfd6384b3080463397c5b619b3626b70)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:22.842849 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444d8a5f-128f-44fd-b942-556ce02539de None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:22.845215 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 47/96] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 6 16:02:22 2022] GET /identity/v3/projects/cfd6384b3080463397c5b619b3626b70 => generated 312 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:22.851233 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d588aa46-4c19-47cf-a006-286e85d2d174 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:22.956007 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d588aa46-4c19-47cf-a006-286e85d2d174 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:22.996118 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea401d35-c1a6-4592-9e9a-167359520b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:22.996742 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea401d35-c1a6-4592-9e9a-167359520b3d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:22.997041 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea401d35-c1a6-4592-9e9a-167359520b3d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:22.997312 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea401d35-c1a6-4592-9e9a-167359520b3d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:22.998057 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea401d35-c1a6-4592-9e9a-167359520b3d None admin] RBAC: Adding query filter params (role.id=15252b0b89894c1e8a6bd9a98b1c3ab7, scope.project.id=cfd6384b3080463397c5b619b3626b70, user.id=6962122ffdbe4d6288f017e98d1e8ecb) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:22.998370 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea401d35-c1a6-4592-9e9a-167359520b3d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:22.999832 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea401d35-c1a6-4592-9e9a-167359520b3d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:23.001736 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:23.006584 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 50/97] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 6 16:02:22 2022] GET /identity/v3/role_assignments?user.id=6962122ffdbe4d6288f017e98d1e8ecb&scope.project.id=cfd6384b3080463397c5b619b3626b70&role.id=15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:23.522865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b695134d-618b-496f-8d11-f1366b553d38 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:23.523626 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b695134d-618b-496f-8d11-f1366b553d38 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:23.523626 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b695134d-618b-496f-8d11-f1366b553d38 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:23.525098 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 48/98] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:02:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:02:23.530503 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcbf980-629f-4597-9a1a-1c3967b011fd None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:23.531014 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcbf980-629f-4597-9a1a-1c3967b011fd None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:23.531014 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcbf980-629f-4597-9a1a-1c3967b011fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:23.559973 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-1fcbf980-629f-4597-9a1a-1c3967b011fd None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:23.561447 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1fcbf980-629f-4597-9a1a-1c3967b011fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:23.647015 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 51/99] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:02:23.778764 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c3a53d-3a4e-402a-91b3-79199da351dc None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:23.779046 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c3a53d-3a4e-402a-91b3-79199da351dc None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:23.779320 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c3a53d-3a4e-402a-91b3-79199da351dc None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:23.780729 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 49/100] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:02:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:02:23.786269 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb533d5f-0a38-4dd6-9343-62d0602ab37d None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:23.786731 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb533d5f-0a38-4dd6-9343-62d0602ab37d None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:23.787138 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb533d5f-0a38-4dd6-9343-62d0602ab37d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:23.813940 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-eb533d5f-0a38-4dd6-9343-62d0602ab37d None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:23.815383 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-eb533d5f-0a38-4dd6-9343-62d0602ab37d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:23.911370 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 52/101] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:02:24.066385 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c3a53d-3a4e-402a-91b3-79199da351dc None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:24.100636 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:24.199655 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c3a53d-3a4e-402a-91b3-79199da351dc None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:24.241993 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ea3a9e-6d85-4b63-92ff-7e988a37a9de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:24.243014 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea3a9e-6d85-4b63-92ff-7e988a37a9de None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:24.243511 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea3a9e-6d85-4b63-92ff-7e988a37a9de None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:24.243973 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea3a9e-6d85-4b63-92ff-7e988a37a9de None admin] PATH_INFO: `/v3/domains/default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:24.247068 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb533d5f-0a38-4dd6-9343-62d0602ab37d None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:24.262896 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ea3a9e-6d85-4b63-92ff-7e988a37a9de None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:24.266492 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ea3a9e-6d85-4b63-92ff-7e988a37a9de None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:24.270209 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 50/102] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:02:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:24.276554 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ea3a9e-6d85-4b63-92ff-7e988a37a9de None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:24.367121 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:24.414439 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb533d5f-0a38-4dd6-9343-62d0602ab37d None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:24.458794 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031047bf-e074-4cd9-bbf4-ff3362fad097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:24.459444 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031047bf-e074-4cd9-bbf4-ff3362fad097 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:24.459722 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031047bf-e074-4cd9-bbf4-ff3362fad097 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:24.459991 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031047bf-e074-4cd9-bbf4-ff3362fad097 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:24.481186 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031047bf-e074-4cd9-bbf4-ff3362fad097 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:02:24.486062 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-031047bf-e074-4cd9-bbf4-ff3362fad097 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 16:02:24.486062 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:24.486062 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:24.486062 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:24.486062 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:24.486062 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:24.486062 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:24.486062 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:24.486062 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:24.486062 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:24.486062 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:24.486062 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:24.486062 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:02:24.486062 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:02:24.486062 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:02:24.486062 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:02:24.486883 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:02:24.486883 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:02:24.486883 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:02:24.486883 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:02:24.486883 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:24.486883 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:24.486883 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:24.486883 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:24.486883 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:24.486883 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:24.486883 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:24.486883 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:24.486883 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:24.486883 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:24.486883 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:24.486883 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:24.487648 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:24.487648 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:24.487648 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:24.487648 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:24.487648 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:02:24.487648 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:02:24.487648 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:02:24.487648 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:02:24.487648 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:02:24.487648 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:02:24.487648 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 16:02:24.487648 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:02:24.487648 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 53/103] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 6 16:02:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:02:24.493699 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031047bf-e074-4cd9-bbf4-ff3362fad097 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:24.499570 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ea3a9e-6d85-4b63-92ff-7e988a37a9de None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:24.510038 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:24.560580 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd03101b-b40a-4a51-91c8-2382dc39adef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:24.561373 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd03101b-b40a-4a51-91c8-2382dc39adef None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:24.561626 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd03101b-b40a-4a51-91c8-2382dc39adef None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:24.561897 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd03101b-b40a-4a51-91c8-2382dc39adef None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:24.562715 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd03101b-b40a-4a51-91c8-2382dc39adef None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:24.564199 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd03101b-b40a-4a51-91c8-2382dc39adef None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:24.619322 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 51/104] 10.222.0.43 () {60 vars in 1141 bytes} [Tue Sep 6 16:02:24 2022] POST /identity/v3/projects => generated 302 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:02:24.691566 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031047bf-e074-4cd9-bbf4-ff3362fad097 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:24.717964 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:24.738977 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0e5548-d64f-46ec-a6ab-242c0220c61f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:24.739596 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0e5548-d64f-46ec-a6ab-242c0220c61f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:24.739878 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0e5548-d64f-46ec-a6ab-242c0220c61f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:24.740149 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0e5548-d64f-46ec-a6ab-242c0220c61f None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:24.740860 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0e5548-d64f-46ec-a6ab-242c0220c61f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:24.741212 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0e5548-d64f-46ec-a6ab-242c0220c61f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:24.742698 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0e5548-d64f-46ec-a6ab-242c0220c61f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:24.755559 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 54/105] 10.222.0.43 () {56 vars in 1101 bytes} [Tue Sep 6 16:02:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:24.761444 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd03101b-b40a-4a51-91c8-2382dc39adef None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:24.889702 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd03101b-b40a-4a51-91c8-2382dc39adef None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:24.901917 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17a43b-c182-463b-a1ab-3adb9f9a4e62 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:24.902243 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17a43b-c182-463b-a1ab-3adb9f9a4e62 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:24.902472 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17a43b-c182-463b-a1ab-3adb9f9a4e62 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:24.903865 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 55/106] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:02:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:02:24.909778 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f07b70-9c9b-4da5-8bfb-2e3bcea847bd None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:24.910066 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f07b70-9c9b-4da5-8bfb-2e3bcea847bd None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:24.910320 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f07b70-9c9b-4da5-8bfb-2e3bcea847bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:24.941858 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00c7110-4302-4fbf-b18f-1c935e39bb02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:24.942509 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c7110-4302-4fbf-b18f-1c935e39bb02 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:24.942789 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c7110-4302-4fbf-b18f-1c935e39bb02 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:24.943060 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c7110-4302-4fbf-b18f-1c935e39bb02 None admin] PATH_INFO: `/v3/users/4d3613f1e41b4d6c8dd2e2b72002ad79` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:24.945433 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-d5f07b70-9c9b-4da5-8bfb-2e3bcea847bd None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:24.946670 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d5f07b70-9c9b-4da5-8bfb-2e3bcea847bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:24.962982 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00c7110-4302-4fbf-b18f-1c935e39bb02 None admin] RBAC: Authorizing `identity:get_user(user_id=4d3613f1e41b4d6c8dd2e2b72002ad79)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:24.966814 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00c7110-4302-4fbf-b18f-1c935e39bb02 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:24.966814 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 52/107] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 6 16:02:24 2022] GET /identity/v3/users/4d3613f1e41b4d6c8dd2e2b72002ad79 => generated 289 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:24.972632 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00c7110-4302-4fbf-b18f-1c935e39bb02 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:24.995609 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:25.069562 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 56/108] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:02:25.071248 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff1d9d3-f37a-4c25-9c77-698a7ccc0b2a None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:25.072343 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff1d9d3-f37a-4c25-9c77-698a7ccc0b2a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:25.072624 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff1d9d3-f37a-4c25-9c77-698a7ccc0b2a None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:25.074363 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 57/109] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:02:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:02:25.079204 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8679244-633f-4785-a754-6f7762f0106f None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:25.079484 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8679244-633f-4785-a754-6f7762f0106f None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:25.079761 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8679244-633f-4785-a754-6f7762f0106f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:25.105110 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-f8679244-633f-4785-a754-6f7762f0106f None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:25.106333 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f8679244-633f-4785-a754-6f7762f0106f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:25.116510 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00c7110-4302-4fbf-b18f-1c935e39bb02 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:25.164512 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d85a2e-9d0c-4991-9601-716dda5a63f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:25.165509 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d85a2e-9d0c-4991-9601-716dda5a63f3 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:25.165998 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d85a2e-9d0c-4991-9601-716dda5a63f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:25.166472 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d85a2e-9d0c-4991-9601-716dda5a63f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:25.234852 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 58/110] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 16:02:25.236091 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d85a2e-9d0c-4991-9601-716dda5a63f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:25.237814 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d85a2e-9d0c-4991-9601-716dda5a63f3 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:25.259461 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:25.329180 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4652b9-6c29-41d3-8cc5-4ca0f28b59eb None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:25.329859 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4652b9-6c29-41d3-8cc5-4ca0f28b59eb None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:25.330319 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4652b9-6c29-41d3-8cc5-4ca0f28b59eb None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:25.331915 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 59/111] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:02:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 16:02:25.337956 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516b4d68-1e8a-4bd9-af17-88ae4ca5c160 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:25.338460 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516b4d68-1e8a-4bd9-af17-88ae4ca5c160 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:25.338923 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516b4d68-1e8a-4bd9-af17-88ae4ca5c160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:25.339699 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 53/112] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 6 16:02:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 06 16:02:25.345362 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d85a2e-9d0c-4991-9601-716dda5a63f3 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:25.388734 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-516b4d68-1e8a-4bd9-af17-88ae4ca5c160 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:25.390302 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-516b4d68-1e8a-4bd9-af17-88ae4ca5c160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:25.475027 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:25.514952 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d85a2e-9d0c-4991-9601-716dda5a63f3 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:25.535981 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:25.536906 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 60/113] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:02:25.574358 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cede075b-d8c6-4b24-a370-f50b370dd9da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:25.575022 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cede075b-d8c6-4b24-a370-f50b370dd9da None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:25.575284 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cede075b-d8c6-4b24-a370-f50b370dd9da None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:25.575552 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cede075b-d8c6-4b24-a370-f50b370dd9da None admin] PATH_INFO: `/v3/projects/5e7a87d573a8446bbcbf9e28149a392f` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:25.576670 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cede075b-d8c6-4b24-a370-f50b370dd9da None admin] RBAC: Authorizing `identity:get_project(project_id=5e7a87d573a8446bbcbf9e28149a392f)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:25.578153 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cede075b-d8c6-4b24-a370-f50b370dd9da None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:25.579802 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 54/114] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 6 16:02:25 2022] GET /identity/v3/projects/5e7a87d573a8446bbcbf9e28149a392f => generated 298 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:25.586491 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516b4d68-1e8a-4bd9-af17-88ae4ca5c160 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:25.611831 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cede075b-d8c6-4b24-a370-f50b370dd9da None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:25.741718 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516b4d68-1e8a-4bd9-af17-88ae4ca5c160 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:25.784680 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cede075b-d8c6-4b24-a370-f50b370dd9da None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:25.796093 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d678ddf2-9c8c-43d4-8fae-be142be7b26f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:25.796725 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d678ddf2-9c8c-43d4-8fae-be142be7b26f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:25.797034 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d678ddf2-9c8c-43d4-8fae-be142be7b26f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:25.797308 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d678ddf2-9c8c-43d4-8fae-be142be7b26f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:25.798040 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d678ddf2-9c8c-43d4-8fae-be142be7b26f None admin] RBAC: Adding query filter params (role.id=4908d6a02c4049e88249e9afbf7e2e9f, scope.project.id=5e7a87d573a8446bbcbf9e28149a392f, user.id=4d3613f1e41b4d6c8dd2e2b72002ad79) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:25.798350 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d678ddf2-9c8c-43d4-8fae-be142be7b26f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:25.799831 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d678ddf2-9c8c-43d4-8fae-be142be7b26f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:25.809296 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 61/115] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 6 16:02:25 2022] GET /identity/v3/role_assignments?user.id=4d3613f1e41b4d6c8dd2e2b72002ad79&scope.project.id=5e7a87d573a8446bbcbf9e28149a392f&role.id=4908d6a02c4049e88249e9afbf7e2e9f => generated 259 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:25.810102 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d678ddf2-9c8c-43d4-8fae-be142be7b26f None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:25.853685 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361492e5-06b3-4ef6-9f00-a4b05ce7733b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:25.854346 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361492e5-06b3-4ef6-9f00-a4b05ce7733b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:25.855097 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361492e5-06b3-4ef6-9f00-a4b05ce7733b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:25.855556 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361492e5-06b3-4ef6-9f00-a4b05ce7733b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:25.877487 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361492e5-06b3-4ef6-9f00-a4b05ce7733b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:02:25.882394 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:25.887052 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-361492e5-06b3-4ef6-9f00-a4b05ce7733b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:02:25.887052 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:25.887052 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:25.887052 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:25.887052 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:25.887052 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:25.887052 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:25.887052 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:25.887052 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:25.887052 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:25.887052 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:25.887052 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:25.887052 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:02:25.887052 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:02:25.887052 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:02:25.887052 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:02:25.888097 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:02:25.888097 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:02:25.888097 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:02:25.888097 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:02:25.888097 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:25.888097 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:25.888097 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:25.888097 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:25.888097 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:25.888097 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:25.888097 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:25.888097 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:25.888097 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:25.888097 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:25.888097 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:25.888097 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:25.889022 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:25.889022 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:25.889022 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:25.889022 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:25.889022 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:02:25.889022 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:02:25.889022 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:02:25.889022 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:02:25.889022 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:02:25.889022 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:02:25.889022 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:02:25.889022 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:02:25.891429 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 55/116] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 6 16:02:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:02:25.892856 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361492e5-06b3-4ef6-9f00-a4b05ce7733b None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:25.962543 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:25.997603 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d678ddf2-9c8c-43d4-8fae-be142be7b26f None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:26.058624 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f25c79-ff90-4e19-8666-109a439f1ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:26.059456 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f25c79-ff90-4e19-8666-109a439f1ae2 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:26.059866 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f25c79-ff90-4e19-8666-109a439f1ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:26.060233 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f25c79-ff90-4e19-8666-109a439f1ae2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:26.071117 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f25c79-ff90-4e19-8666-109a439f1ae2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:02:26.073753 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361492e5-06b3-4ef6-9f00-a4b05ce7733b None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:26.076824 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-04f25c79-ff90-4e19-8666-109a439f1ae2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 16:02:26.076824 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:26.076824 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:26.076824 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:26.076824 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:26.076824 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:26.076824 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:26.076824 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:26.076824 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:26.076824 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:26.076824 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:26.076824 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:26.076824 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:02:26.076824 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:02:26.076824 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:02:26.076824 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:02:26.077658 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:02:26.077658 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:02:26.077658 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:02:26.077658 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:02:26.077658 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:26.077658 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:26.077658 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:26.077658 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:26.077658 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:26.077658 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:26.077658 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:26.077658 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:26.077658 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:26.077658 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:26.077658 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:26.077658 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:26.078407 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:26.078407 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:26.078407 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:26.078407 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:26.078407 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:02:26.078407 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:02:26.078407 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:02:26.078407 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:02:26.078407 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:02:26.078407 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:02:26.078407 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 16:02:26.078407 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:02:26.078407 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 62/117] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 6 16:02:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:02:26.079342 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f25c79-ff90-4e19-8666-109a439f1ae2 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:26.118785 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf76179-09b4-47da-8b91-ec2a7bca3f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:26.119434 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf76179-09b4-47da-8b91-ec2a7bca3f4b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:26.119991 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf76179-09b4-47da-8b91-ec2a7bca3f4b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:26.120282 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf76179-09b4-47da-8b91-ec2a7bca3f4b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:26.134970 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf76179-09b4-47da-8b91-ec2a7bca3f4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:02:26.141098 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-fcf76179-09b4-47da-8b91-ec2a7bca3f4b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 16:02:26.141098 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:26.141098 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:26.141098 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:26.141098 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:26.141098 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:26.141098 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:26.141098 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:26.141098 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:26.141098 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:26.141098 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:26.141098 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:26.141098 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:02:26.141098 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:02:26.141098 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:02:26.141098 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:02:26.143157 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:02:26.143157 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:02:26.143157 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:02:26.143157 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:02:26.143157 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:26.143157 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:26.143157 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:26.143157 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:26.143157 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:26.143157 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:26.143157 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:26.143157 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:26.143157 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:26.143157 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:26.143157 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:26.143157 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:26.144353 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:26.144353 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:26.144353 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:26.144353 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:26.144353 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:02:26.144353 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:02:26.144353 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:02:26.144353 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:02:26.144353 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:02:26.144353 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:02:26.144353 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 16:02:26.144353 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:02:26.144353 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 56/118] 10.222.0.43 () {56 vars in 1114 bytes} [Tue Sep 6 16:02:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:02:26.144353 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733687f0-6c9e-4ac3-97c7-86ae31ac8250 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:26.145299 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733687f0-6c9e-4ac3-97c7-86ae31ac8250 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:26.145299 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733687f0-6c9e-4ac3-97c7-86ae31ac8250 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:26.149976 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 57/119] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:02:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:02:26.149976 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733687f0-6c9e-4ac3-97c7-86ae31ac8250 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:26.191200 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:26.201034 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f25c79-ff90-4e19-8666-109a439f1ae2 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:26.237743 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9e0593-9c7d-4a51-b3c5-dc2eb8a09e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:26.238548 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9e0593-9c7d-4a51-b3c5-dc2eb8a09e41 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:26.238927 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9e0593-9c7d-4a51-b3c5-dc2eb8a09e41 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:26.239297 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9e0593-9c7d-4a51-b3c5-dc2eb8a09e41 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:26.240091 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9e0593-9c7d-4a51-b3c5-dc2eb8a09e41 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:26.240491 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9e0593-9c7d-4a51-b3c5-dc2eb8a09e41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:26.242150 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9e0593-9c7d-4a51-b3c5-dc2eb8a09e41 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:26.257913 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 63/120] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 6 16:02:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:26.258791 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9e0593-9c7d-4a51-b3c5-dc2eb8a09e41 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:26.282567 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:26.287713 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733687f0-6c9e-4ac3-97c7-86ae31ac8250 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:26.324960 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831e3271-5474-4bed-ac6a-89fda1a1d0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:26.325792 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e3271-5474-4bed-ac6a-89fda1a1d0e2 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:26.325934 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e3271-5474-4bed-ac6a-89fda1a1d0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:26.326106 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e3271-5474-4bed-ac6a-89fda1a1d0e2 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:26.326824 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831e3271-5474-4bed-ac6a-89fda1a1d0e2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:26.327123 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831e3271-5474-4bed-ac6a-89fda1a1d0e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:26.328628 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831e3271-5474-4bed-ac6a-89fda1a1d0e2 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:26.337919 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 58/121] 10.222.0.43 () {56 vars in 1101 bytes} [Tue Sep 6 16:02:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:26.339353 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc895c8a-7490-42a0-a627-2f7c219f918b None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:26.339800 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc895c8a-7490-42a0-a627-2f7c219f918b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:26.340151 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc895c8a-7490-42a0-a627-2f7c219f918b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:26.362281 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-cc895c8a-7490-42a0-a627-2f7c219f918b None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:26.363456 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-cc895c8a-7490-42a0-a627-2f7c219f918b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:26.390442 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9e0593-9c7d-4a51-b3c5-dc2eb8a09e41 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:26.433498 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 59/122] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:02:26.440995 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc895c8a-7490-42a0-a627-2f7c219f918b None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:26.441098 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf287442-8bf3-4ca6-81e6-1765ba357119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:26.441098 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf287442-8bf3-4ca6-81e6-1765ba357119 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:26.441098 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf287442-8bf3-4ca6-81e6-1765ba357119 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:26.441098 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf287442-8bf3-4ca6-81e6-1765ba357119 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:26.441453 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf287442-8bf3-4ca6-81e6-1765ba357119 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:26.441453 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf287442-8bf3-4ca6-81e6-1765ba357119 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:26.442552 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf287442-8bf3-4ca6-81e6-1765ba357119 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:26.462101 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 64/123] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 6 16:02:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:26.463764 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf287442-8bf3-4ca6-81e6-1765ba357119 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:26.496666 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:26.590945 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:26.612350 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc895c8a-7490-42a0-a627-2f7c219f918b None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:26.655700 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf287442-8bf3-4ca6-81e6-1765ba357119 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:26.657137 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6755c8a-956a-40dc-8242-14ef438b73f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:26.658074 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6755c8a-956a-40dc-8242-14ef438b73f7 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:26.658363 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6755c8a-956a-40dc-8242-14ef438b73f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:26.658639 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6755c8a-956a-40dc-8242-14ef438b73f7 None admin] PATH_INFO: `/v3/users/824c9880f41b4962b83243abf75b8162` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:26.661196 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6755c8a-956a-40dc-8242-14ef438b73f7 None admin] RBAC: Authorizing `identity:get_user(user_id=824c9880f41b4962b83243abf75b8162)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:26.662670 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6755c8a-956a-40dc-8242-14ef438b73f7 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:26.664605 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 60/124] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 6 16:02:26 2022] GET /identity/v3/users/824c9880f41b4962b83243abf75b8162 => generated 247 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:26.672989 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6755c8a-956a-40dc-8242-14ef438b73f7 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:26.732781 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d361457-3009-4bdd-b834-8380766a1339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:26.733432 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d361457-3009-4bdd-b834-8380766a1339 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:26.733713 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d361457-3009-4bdd-b834-8380766a1339 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:26.733989 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d361457-3009-4bdd-b834-8380766a1339 None admin] PATH_INFO: `/v3/users/6962122ffdbe4d6288f017e98d1e8ecb` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:26.737314 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d361457-3009-4bdd-b834-8380766a1339 None admin] RBAC: Authorizing `identity:get_user(user_id=6962122ffdbe4d6288f017e98d1e8ecb)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:26.738807 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d361457-3009-4bdd-b834-8380766a1339 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:26.741776 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 65/125] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 6 16:02:26 2022] GET /identity/v3/users/6962122ffdbe4d6288f017e98d1e8ecb => generated 275 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:26.742628 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d361457-3009-4bdd-b834-8380766a1339 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:26.830746 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6755c8a-956a-40dc-8242-14ef438b73f7 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:26.832785 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:26.849875 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d361457-3009-4bdd-b834-8380766a1339 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:26.867778 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7a2e9d-7519-48ae-a1d1-b1b999504151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:26.868827 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7a2e9d-7519-48ae-a1d1-b1b999504151 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:26.869303 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7a2e9d-7519-48ae-a1d1-b1b999504151 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:26.869761 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7a2e9d-7519-48ae-a1d1-b1b999504151 None admin] PATH_INFO: `/v3/users/6962122ffdbe4d6288f017e98d1e8ecb` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:26.871170 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7a2e9d-7519-48ae-a1d1-b1b999504151 None admin] RBAC: Authorizing `identity:get_user(user_id=6962122ffdbe4d6288f017e98d1e8ecb)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:26.872884 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7a2e9d-7519-48ae-a1d1-b1b999504151 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:26.874886 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 61/126] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 6 16:02:26 2022] GET /identity/v3/users/6962122ffdbe4d6288f017e98d1e8ecb => generated 275 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:26.876003 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7a2e9d-7519-48ae-a1d1-b1b999504151 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:26.888911 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e3dc71-9855-4024-8eb0-13cea2b32a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:26.889611 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e3dc71-9855-4024-8eb0-13cea2b32a81 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:26.889895 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e3dc71-9855-4024-8eb0-13cea2b32a81 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:26.890174 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e3dc71-9855-4024-8eb0-13cea2b32a81 None admin] PATH_INFO: `/v3/projects/5e7a87d573a8446bbcbf9e28149a392f` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:26.891363 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e3dc71-9855-4024-8eb0-13cea2b32a81 None admin] RBAC: Authorizing `identity:get_project(project_id=5e7a87d573a8446bbcbf9e28149a392f)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:26.892888 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e3dc71-9855-4024-8eb0-13cea2b32a81 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:26.894727 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 66/127] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 6 16:02:26 2022] GET /identity/v3/projects/5e7a87d573a8446bbcbf9e28149a392f => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:26.895359 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e3dc71-9855-4024-8eb0-13cea2b32a81 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:26.907709 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:26.984274 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7a2e9d-7519-48ae-a1d1-b1b999504151 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:27.014948 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e3dc71-9855-4024-8eb0-13cea2b32a81 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:27.021756 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20164291-8ff0-4de1-9787-82c391849a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:27.022723 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20164291-8ff0-4de1-9787-82c391849a93 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:27.023225 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20164291-8ff0-4de1-9787-82c391849a93 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:27.023688 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20164291-8ff0-4de1-9787-82c391849a93 None admin] PATH_INFO: `/v3/projects/5e7a87d573a8446bbcbf9e28149a392f` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:27.025045 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20164291-8ff0-4de1-9787-82c391849a93 None admin] RBAC: Authorizing `identity:get_project(project_id=5e7a87d573a8446bbcbf9e28149a392f)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:27.026712 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20164291-8ff0-4de1-9787-82c391849a93 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:27.028613 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 62/128] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 6 16:02:26 2022] GET /identity/v3/projects/5e7a87d573a8446bbcbf9e28149a392f => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:27.029664 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20164291-8ff0-4de1-9787-82c391849a93 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:27.052732 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d268fcb-219d-403c-8662-d69d69e71c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:27.053393 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d268fcb-219d-403c-8662-d69d69e71c00 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:27.053676 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d268fcb-219d-403c-8662-d69d69e71c00 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:27.053952 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d268fcb-219d-403c-8662-d69d69e71c00 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:27.066124 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d268fcb-219d-403c-8662-d69d69e71c00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:02:27.072032 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-5d268fcb-219d-403c-8662-d69d69e71c00 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 16:02:27.072032 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:27.072032 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:27.072032 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:27.072032 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:27.072032 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:27.072032 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:27.072032 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:27.072032 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:27.072032 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:27.072032 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:27.072032 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:27.072032 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:02:27.072032 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:02:27.072032 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:02:27.072032 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:02:27.072922 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:02:27.072922 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:02:27.072922 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:02:27.072922 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:02:27.072922 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:27.072922 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:27.072922 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:27.072922 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:27.072922 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:27.072922 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:27.072922 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:27.072922 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:27.072922 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:27.072922 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:27.072922 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:27.072922 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:27.078715 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:27.078715 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:27.078715 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:27.078715 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:27.078715 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:02:27.078715 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:02:27.078715 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:02:27.078715 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:02:27.078715 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:02:27.078715 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:02:27.078715 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 16:02:27.078715 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:02:27.078715 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 67/129] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 6 16:02:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:02:27.078715 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d268fcb-219d-403c-8662-d69d69e71c00 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:27.098931 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:27.137104 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:27.144635 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20164291-8ff0-4de1-9787-82c391849a93 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:27.180443 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d97ebe-c802-4cf3-a447-c0fe015ab174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:27.181136 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d97ebe-c802-4cf3-a447-c0fe015ab174 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:27.181419 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d97ebe-c802-4cf3-a447-c0fe015ab174 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:27.181694 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d97ebe-c802-4cf3-a447-c0fe015ab174 None admin] PATH_INFO: `/v3/projects/5e7a87d573a8446bbcbf9e28149a392f` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:27.182811 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d97ebe-c802-4cf3-a447-c0fe015ab174 None admin] RBAC: Authorizing `identity:get_project(project_id=5e7a87d573a8446bbcbf9e28149a392f)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:27.184265 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d97ebe-c802-4cf3-a447-c0fe015ab174 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:27.186164 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 63/130] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 6 16:02:27 2022] GET /identity/v3/projects/5e7a87d573a8446bbcbf9e28149a392f => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:27.186699 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d97ebe-c802-4cf3-a447-c0fe015ab174 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:27.193572 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d268fcb-219d-403c-8662-d69d69e71c00 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:27.229194 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd311c03-bb5a-4276-8b51-36de05b2ce88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:27.230253 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd311c03-bb5a-4276-8b51-36de05b2ce88 None admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:27.230541 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd311c03-bb5a-4276-8b51-36de05b2ce88 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:27.230823 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd311c03-bb5a-4276-8b51-36de05b2ce88 None admin] PATH_INFO: `/v3/projects/5e7a87d573a8446bbcbf9e28149a392f/users/824c9880f41b4962b83243abf75b8162/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:27.233086 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd311c03-bb5a-4276-8b51-36de05b2ce88 None admin] RBAC: Authorizing `identity:create_grant(project_id=5e7a87d573a8446bbcbf9e28149a392f, user_id=824c9880f41b4962b83243abf75b8162, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:27.234586 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd311c03-bb5a-4276-8b51-36de05b2ce88 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:27.248184 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 68/131] 10.222.0.43 () {58 vars in 1439 bytes} [Tue Sep 6 16:02:27 2022] PUT /identity/v3/projects/5e7a87d573a8446bbcbf9e28149a392f/users/824c9880f41b4962b83243abf75b8162/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:02:27.249056 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd311c03-bb5a-4276-8b51-36de05b2ce88 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:27.326922 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d97ebe-c802-4cf3-a447-c0fe015ab174 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:27.350299 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:27.373679 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7161e3-b0a4-41e0-a93a-d60802371b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:27.374546 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7161e3-b0a4-41e0-a93a-d60802371b13 None admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:27.374931 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7161e3-b0a4-41e0-a93a-d60802371b13 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:27.375440 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7161e3-b0a4-41e0-a93a-d60802371b13 None admin] PATH_INFO: `/v3/projects/5e7a87d573a8446bbcbf9e28149a392f/users/6962122ffdbe4d6288f017e98d1e8ecb/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:27.378429 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7161e3-b0a4-41e0-a93a-d60802371b13 None admin] RBAC: Authorizing `identity:create_grant(project_id=5e7a87d573a8446bbcbf9e28149a392f, user_id=6962122ffdbe4d6288f017e98d1e8ecb, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:27.381000 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7161e3-b0a4-41e0-a93a-d60802371b13 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:27.382966 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd311c03-bb5a-4276-8b51-36de05b2ce88 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:27.406177 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 64/132] 10.222.0.43 () {58 vars in 1439 bytes} [Tue Sep 6 16:02:27 2022] PUT /identity/v3/projects/5e7a87d573a8446bbcbf9e28149a392f/users/6962122ffdbe4d6288f017e98d1e8ecb/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:02:27.408405 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b87001-fb44-42da-b53e-cf4f2995f978 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:27.408926 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b87001-fb44-42da-b53e-cf4f2995f978 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:27.409365 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b87001-fb44-42da-b53e-cf4f2995f978 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:27.411079 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 65/133] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:02:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:02:27.412206 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b87001-fb44-42da-b53e-cf4f2995f978 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:27.443527 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb96ce8f-348a-4e1a-84ed-c40f031d8486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:27.444345 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb96ce8f-348a-4e1a-84ed-c40f031d8486 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:27.445053 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb96ce8f-348a-4e1a-84ed-c40f031d8486 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:27.445363 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb96ce8f-348a-4e1a-84ed-c40f031d8486 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:27.446144 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb96ce8f-348a-4e1a-84ed-c40f031d8486 None admin] RBAC: Adding query filter params (name=member) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:27.446464 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb96ce8f-348a-4e1a-84ed-c40f031d8486 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:27.447912 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb96ce8f-348a-4e1a-84ed-c40f031d8486 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:27.448702 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:27.458824 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 69/134] 10.222.0.43 () {56 vars in 1101 bytes} [Tue Sep 6 16:02:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:27.460368 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e110c19e-6069-496d-99fb-ebcb613a4b24 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:27.462637 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e110c19e-6069-496d-99fb-ebcb613a4b24 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:27.463478 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e110c19e-6069-496d-99fb-ebcb613a4b24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:27.488945 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-e110c19e-6069-496d-99fb-ebcb613a4b24 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:27.490165 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e110c19e-6069-496d-99fb-ebcb613a4b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:27.553246 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b87001-fb44-42da-b53e-cf4f2995f978 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:27.593649 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0df1e2-7ef2-4ecb-aa47-da9c320eec7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:27.594485 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0df1e2-7ef2-4ecb-aa47-da9c320eec7e None admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:27.594863 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0df1e2-7ef2-4ecb-aa47-da9c320eec7e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:27.595715 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0df1e2-7ef2-4ecb-aa47-da9c320eec7e None admin] PATH_INFO: `/v3/projects/5e7a87d573a8446bbcbf9e28149a392f/users/6962122ffdbe4d6288f017e98d1e8ecb/roles/61df19b017494bdcadc9a5e9d8bb9e52` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:27.598285 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0df1e2-7ef2-4ecb-aa47-da9c320eec7e None admin] RBAC: Authorizing `identity:create_grant(project_id=5e7a87d573a8446bbcbf9e28149a392f, user_id=6962122ffdbe4d6288f017e98d1e8ecb, role_id=61df19b017494bdcadc9a5e9d8bb9e52)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:27.599892 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0df1e2-7ef2-4ecb-aa47-da9c320eec7e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:27.619552 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 66/135] 10.222.0.43 () {58 vars in 1439 bytes} [Tue Sep 6 16:02:27 2022] PUT /identity/v3/projects/5e7a87d573a8446bbcbf9e28149a392f/users/6962122ffdbe4d6288f017e98d1e8ecb/roles/61df19b017494bdcadc9a5e9d8bb9e52 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:02:27.620241 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0df1e2-7ef2-4ecb-aa47-da9c320eec7e None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:27.679185 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 70/136] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:02:27.736886 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:27.758203 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0df1e2-7ef2-4ecb-aa47-da9c320eec7e None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:27.807309 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2790381e-55ee-42b3-9670-43fef92bbe45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:27.808425 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2790381e-55ee-42b3-9670-43fef92bbe45 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:27.808711 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2790381e-55ee-42b3-9670-43fef92bbe45 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:27.809637 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2790381e-55ee-42b3-9670-43fef92bbe45 None admin] PATH_INFO: `/v3/users/6962122ffdbe4d6288f017e98d1e8ecb` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:27.810881 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2790381e-55ee-42b3-9670-43fef92bbe45 None admin] RBAC: Authorizing `identity:get_user(user_id=6962122ffdbe4d6288f017e98d1e8ecb)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:27.812434 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2790381e-55ee-42b3-9670-43fef92bbe45 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:27.814092 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 67/137] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 6 16:02:27 2022] GET /identity/v3/users/6962122ffdbe4d6288f017e98d1e8ecb => generated 275 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:27.820148 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e110c19e-6069-496d-99fb-ebcb613a4b24 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:27.912389 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:27.925313 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b25eec-1d88-4aab-be8a-3db11710efaa None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:27.925611 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b25eec-1d88-4aab-be8a-3db11710efaa None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:27.925868 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b25eec-1d88-4aab-be8a-3db11710efaa None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:27.927351 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 68/138] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:02:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 16:02:27.933025 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef4e6ed-f135-409d-b987-dd169da76985 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:27.933578 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef4e6ed-f135-409d-b987-dd169da76985 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:27.934061 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef4e6ed-f135-409d-b987-dd169da76985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:27.955544 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-fef4e6ed-f135-409d-b987-dd169da76985 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:27.957553 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-fef4e6ed-f135-409d-b987-dd169da76985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:27.963171 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e110c19e-6069-496d-99fb-ebcb613a4b24 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:28.025479 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f629789-0a86-47df-8d43-736b4d35fbd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:28.026136 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f629789-0a86-47df-8d43-736b4d35fbd9 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:28.026412 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f629789-0a86-47df-8d43-736b4d35fbd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:28.026680 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f629789-0a86-47df-8d43-736b4d35fbd9 None admin] PATH_INFO: `/v3/projects/cfd6384b3080463397c5b619b3626b70` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:28.033461 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f629789-0a86-47df-8d43-736b4d35fbd9 None admin] RBAC: Authorizing `identity:get_project(project_id=cfd6384b3080463397c5b619b3626b70)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:28.034953 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f629789-0a86-47df-8d43-736b4d35fbd9 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:28.036877 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 71/139] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 6 16:02:27 2022] GET /identity/v3/projects/cfd6384b3080463397c5b619b3626b70 => generated 312 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:28.042580 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f629789-0a86-47df-8d43-736b4d35fbd9 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:28.072963 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 69/140] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 16:02:28.160184 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f629789-0a86-47df-8d43-736b4d35fbd9 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:28.175067 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef4e6ed-f135-409d-b987-dd169da76985 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:28.202437 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e906510f-977e-441d-bd04-06f32d687473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:28.203034 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906510f-977e-441d-bd04-06f32d687473 None admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:28.203306 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906510f-977e-441d-bd04-06f32d687473 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:28.203575 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906510f-977e-441d-bd04-06f32d687473 None admin] PATH_INFO: `/v3/projects/cfd6384b3080463397c5b619b3626b70/users/6962122ffdbe4d6288f017e98d1e8ecb/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:28.210063 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e906510f-977e-441d-bd04-06f32d687473 None admin] RBAC: Authorizing `identity:create_grant(project_id=cfd6384b3080463397c5b619b3626b70, user_id=6962122ffdbe4d6288f017e98d1e8ecb, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:28.211595 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e906510f-977e-441d-bd04-06f32d687473 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:28.213940 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:28.242585 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 72/141] 10.222.0.43 () {58 vars in 1439 bytes} [Tue Sep 6 16:02:28 2022] PUT /identity/v3/projects/cfd6384b3080463397c5b619b3626b70/users/6962122ffdbe4d6288f017e98d1e8ecb/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:02:28.322999 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:28.358493 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef4e6ed-f135-409d-b987-dd169da76985 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:28.411520 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922cfc82-65fb-40c9-bf6f-a22fe79d1aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:28.412214 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922cfc82-65fb-40c9-bf6f-a22fe79d1aa0 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:28.412495 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922cfc82-65fb-40c9-bf6f-a22fe79d1aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:28.412788 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922cfc82-65fb-40c9-bf6f-a22fe79d1aa0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:28.417572 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922cfc82-65fb-40c9-bf6f-a22fe79d1aa0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:28.419083 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922cfc82-65fb-40c9-bf6f-a22fe79d1aa0 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:28.420890 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 70/142] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:02:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:02:28.426535 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e906510f-977e-441d-bd04-06f32d687473 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:28.471527 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922cfc82-65fb-40c9-bf6f-a22fe79d1aa0 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:28.570653 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e906510f-977e-441d-bd04-06f32d687473 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:28.609779 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688ab4c6-93df-4fe7-9da7-69f35656303a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:28.610880 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ab4c6-93df-4fe7-9da7-69f35656303a None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:28.611357 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ab4c6-93df-4fe7-9da7-69f35656303a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:28.611829 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ab4c6-93df-4fe7-9da7-69f35656303a None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:28.612764 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688ab4c6-93df-4fe7-9da7-69f35656303a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:28.614763 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688ab4c6-93df-4fe7-9da7-69f35656303a None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:28.644593 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 73/143] 10.222.0.43 () {60 vars in 1132 bytes} [Tue Sep 6 16:02:28 2022] POST /identity/v3/users => generated 283 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:02:28.667481 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922cfc82-65fb-40c9-bf6f-a22fe79d1aa0 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:28.712659 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23685283-1544-4a7c-bdb2-989c88c1f52f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:28.713884 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23685283-1544-4a7c-bdb2-989c88c1f52f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:28.714382 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23685283-1544-4a7c-bdb2-989c88c1f52f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:28.714849 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23685283-1544-4a7c-bdb2-989c88c1f52f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:28.745485 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23685283-1544-4a7c-bdb2-989c88c1f52f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:02:28.751919 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-23685283-1544-4a7c-bdb2-989c88c1f52f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 16:02:28.751919 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:28.751919 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:28.751919 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:28.751919 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:28.751919 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:28.751919 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:28.751919 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:28.751919 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:28.751919 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:28.751919 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:28.751919 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:28.751919 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:02:28.751919 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:02:28.751919 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:02:28.751919 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:02:28.752814 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:02:28.752814 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:02:28.752814 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:02:28.752814 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:02:28.752814 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:28.752814 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:28.752814 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:28.752814 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:28.752814 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:28.752814 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:28.752814 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:28.752814 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:28.752814 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:28.752814 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:28.752814 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:28.752814 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:28.753583 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:28.753583 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:28.753583 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:28.753583 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:28.753583 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:02:28.753583 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:02:28.753583 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:02:28.753583 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:02:28.753583 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:02:28.753583 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:02:28.753583 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 16:02:28.753583 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:02:28.756130 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 71/144] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 6 16:02:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:02:28.762172 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688ab4c6-93df-4fe7-9da7-69f35656303a None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:28.840601 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:28.915622 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688ab4c6-93df-4fe7-9da7-69f35656303a None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:28.959653 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48f4fc2-b999-48cb-acfc-ebb8e5db961e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:28.960668 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f4fc2-b999-48cb-acfc-ebb8e5db961e None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:28.961187 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f4fc2-b999-48cb-acfc-ebb8e5db961e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:28.961652 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f4fc2-b999-48cb-acfc-ebb8e5db961e None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:28.962526 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48f4fc2-b999-48cb-acfc-ebb8e5db961e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:28.963038 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48f4fc2-b999-48cb-acfc-ebb8e5db961e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:28.964701 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48f4fc2-b999-48cb-acfc-ebb8e5db961e None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:28.980733 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 74/145] 10.222.0.43 () {56 vars in 1101 bytes} [Tue Sep 6 16:02:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:28.986037 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23685283-1544-4a7c-bdb2-989c88c1f52f None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:28.998297 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:29.115469 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23685283-1544-4a7c-bdb2-989c88c1f52f None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:29.154512 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee8d4e8-9743-464e-928c-3224d7bee6da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:29.155781 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee8d4e8-9743-464e-928c-3224d7bee6da None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:29.156258 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee8d4e8-9743-464e-928c-3224d7bee6da None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:29.156743 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee8d4e8-9743-464e-928c-3224d7bee6da None admin] PATH_INFO: `/v3/users/4d3613f1e41b4d6c8dd2e2b72002ad79` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:29.158109 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee8d4e8-9743-464e-928c-3224d7bee6da None admin] RBAC: Authorizing `identity:get_user(user_id=4d3613f1e41b4d6c8dd2e2b72002ad79)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:29.159779 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee8d4e8-9743-464e-928c-3224d7bee6da None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:29.161633 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 72/146] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 6 16:02:28 2022] GET /identity/v3/users/4d3613f1e41b4d6c8dd2e2b72002ad79 => generated 289 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:29.167389 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48f4fc2-b999-48cb-acfc-ebb8e5db961e None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:29.305464 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48f4fc2-b999-48cb-acfc-ebb8e5db961e None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:29.321342 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:29.349959 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc494cc-cfcd-4347-9313-91907a51761a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:29.350963 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc494cc-cfcd-4347-9313-91907a51761a None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:29.351584 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc494cc-cfcd-4347-9313-91907a51761a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:29.352053 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc494cc-cfcd-4347-9313-91907a51761a None admin] PATH_INFO: `/v3/projects/5e7a87d573a8446bbcbf9e28149a392f` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:29.353462 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc494cc-cfcd-4347-9313-91907a51761a None admin] RBAC: Authorizing `identity:get_project(project_id=5e7a87d573a8446bbcbf9e28149a392f)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:29.355297 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc494cc-cfcd-4347-9313-91907a51761a None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:29.357360 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 75/147] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 6 16:02:29 2022] GET /identity/v3/projects/5e7a87d573a8446bbcbf9e28149a392f => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:29.363186 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee8d4e8-9743-464e-928c-3224d7bee6da None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:29.469526 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:29.501435 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee8d4e8-9743-464e-928c-3224d7bee6da None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:29.544557 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5c7090-0e78-4194-8ed2-8a2c2d2298cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:29.545199 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5c7090-0e78-4194-8ed2-8a2c2d2298cd None admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:29.545478 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5c7090-0e78-4194-8ed2-8a2c2d2298cd None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:29.545746 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5c7090-0e78-4194-8ed2-8a2c2d2298cd None admin] PATH_INFO: `/v3/projects/5e7a87d573a8446bbcbf9e28149a392f/users/4d3613f1e41b4d6c8dd2e2b72002ad79/roles/4908d6a02c4049e88249e9afbf7e2e9f` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:29.547830 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5c7090-0e78-4194-8ed2-8a2c2d2298cd None admin] RBAC: Authorizing `identity:create_grant(project_id=5e7a87d573a8446bbcbf9e28149a392f, user_id=4d3613f1e41b4d6c8dd2e2b72002ad79, role_id=4908d6a02c4049e88249e9afbf7e2e9f)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:29.549393 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5c7090-0e78-4194-8ed2-8a2c2d2298cd None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:29.570784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 73/148] 10.222.0.43 () {58 vars in 1439 bytes} [Tue Sep 6 16:02:29 2022] PUT /identity/v3/projects/5e7a87d573a8446bbcbf9e28149a392f/users/4d3613f1e41b4d6c8dd2e2b72002ad79/roles/4908d6a02c4049e88249e9afbf7e2e9f => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:02:29.932368 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ced8a4-6482-485a-95a2-ca7740757c14 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:29.933357 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ced8a4-6482-485a-95a2-ca7740757c14 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:29.933907 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ced8a4-6482-485a-95a2-ca7740757c14 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:29.935477 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 76/149] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:02:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:02:29.940911 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8fef9c-7438-460e-b8b4-e8b0619de55e None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:29.941337 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8fef9c-7438-460e-b8b4-e8b0619de55e None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:29.941515 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8fef9c-7438-460e-b8b4-e8b0619de55e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:29.969934 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-3b8fef9c-7438-460e-b8b4-e8b0619de55e None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:29.971176 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-3b8fef9c-7438-460e-b8b4-e8b0619de55e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:30.061542 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 74/150] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:02:30.134482 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d686b876-f492-432b-aa48-d11be2522530 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:30.135103 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d686b876-f492-432b-aa48-d11be2522530 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:30.135588 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d686b876-f492-432b-aa48-d11be2522530 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:30.137192 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 77/151] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:02:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:02:30.146677 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e446ed42-4d31-454c-9b47-01aab94fe951 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:30.146961 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e446ed42-4d31-454c-9b47-01aab94fe951 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:30.147236 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e446ed42-4d31-454c-9b47-01aab94fe951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:30.177792 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-e446ed42-4d31-454c-9b47-01aab94fe951 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:30.178999 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e446ed42-4d31-454c-9b47-01aab94fe951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:30.250379 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:30.263629 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 75/152] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:02:30.367866 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d686b876-f492-432b-aa48-d11be2522530 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:30.491391 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d686b876-f492-432b-aa48-d11be2522530 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:30.534490 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57402558-533f-46ea-b1ed-a90c5f7c2009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:30.535518 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57402558-533f-46ea-b1ed-a90c5f7c2009 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:30.535991 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57402558-533f-46ea-b1ed-a90c5f7c2009 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:30.536456 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57402558-533f-46ea-b1ed-a90c5f7c2009 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:30.550500 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57402558-533f-46ea-b1ed-a90c5f7c2009 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:02:30.555394 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-57402558-533f-46ea-b1ed-a90c5f7c2009 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 16:02:30.555394 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:30.555394 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:30.555394 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:30.555394 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:30.555394 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:30.555394 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:30.555394 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:30.555394 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:30.555394 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:30.555394 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:30.555394 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:30.555394 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:02:30.555394 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:02:30.555394 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:02:30.555394 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:02:30.556249 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:02:30.556249 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:02:30.556249 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:02:30.556249 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:02:30.556249 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:30.556249 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:30.556249 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:30.556249 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:30.556249 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:30.556249 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:30.556249 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:30.556249 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:30.556249 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:30.556249 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:30.556249 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:30.556249 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:30.557121 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:30.557121 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:30.557121 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:30.557121 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:30.557121 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:02:30.557121 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:02:30.557121 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:02:30.557121 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:02:30.557121 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:02:30.557121 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:02:30.557121 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 16:02:30.557121 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:02:30.557121 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 78/153] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 6 16:02:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:02:30.565938 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e446ed42-4d31-454c-9b47-01aab94fe951 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:30.632089 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57402558-533f-46ea-b1ed-a90c5f7c2009 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:30.688339 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:30.692286 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e446ed42-4d31-454c-9b47-01aab94fe951 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:30.742213 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c22a3f1-2d82-4076-837b-b991034c908a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:30.742860 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c22a3f1-2d82-4076-837b-b991034c908a None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:30.743132 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c22a3f1-2d82-4076-837b-b991034c908a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:30.743405 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c22a3f1-2d82-4076-837b-b991034c908a None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:30.744092 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c22a3f1-2d82-4076-837b-b991034c908a None admin] RBAC: Adding query filter params (name=member) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:30.744393 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c22a3f1-2d82-4076-837b-b991034c908a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:30.745841 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c22a3f1-2d82-4076-837b-b991034c908a None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:30.754788 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 76/154] 10.222.0.43 () {56 vars in 1101 bytes} [Tue Sep 6 16:02:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:30.756063 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9b000-457a-4f66-9f9d-16a80723e1b6 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:30.756350 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9b000-457a-4f66-9f9d-16a80723e1b6 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:30.756627 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9b000-457a-4f66-9f9d-16a80723e1b6 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:30.762370 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 77/155] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:02:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:02:30.763166 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c9b000-457a-4f66-9f9d-16a80723e1b6 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:30.823878 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57402558-533f-46ea-b1ed-a90c5f7c2009 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:30.874205 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:30.903370 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c8e70e-b4c4-4643-a0b7-cab2580e4b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:30.904420 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c8e70e-b4c4-4643-a0b7-cab2580e4b28 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:30.904898 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c8e70e-b4c4-4643-a0b7-cab2580e4b28 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:30.905183 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c8e70e-b4c4-4643-a0b7-cab2580e4b28 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:30.929289 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c8e70e-b4c4-4643-a0b7-cab2580e4b28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:02:30.935742 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-67c8e70e-b4c4-4643-a0b7-cab2580e4b28 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:02:30.935742 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:30.935742 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:30.935742 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:30.935742 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:30.935742 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:30.935742 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:30.935742 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:30.935742 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:30.935742 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:30.935742 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:30.935742 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:30.935742 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:02:30.935742 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:02:30.935742 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:02:30.935742 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:02:30.937032 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:02:30.937032 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:02:30.937032 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:02:30.937032 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:02:30.937032 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:30.937032 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:30.937032 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:30.937032 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:30.937032 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:30.937032 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:30.937032 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:30.937032 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:30.937032 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:30.937032 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:30.937032 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:30.937032 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:30.937942 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:30.937942 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:30.937942 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:30.937942 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:30.937942 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:02:30.937942 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:02:30.937942 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:02:30.937942 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:02:30.937942 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:02:30.937942 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:02:30.937942 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:02:30.937942 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:02:30.937942 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 79/156] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 6 16:02:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:02:30.938629 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a58bd2c-6d10-4f8b-af18-4177b1d57416 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:30.938629 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a58bd2c-6d10-4f8b-af18-4177b1d57416 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:30.938629 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a58bd2c-6d10-4f8b-af18-4177b1d57416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:30.954377 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c9b000-457a-4f66-9f9d-16a80723e1b6 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:30.981846 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-6a58bd2c-6d10-4f8b-af18-4177b1d57416 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:30.983077 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6a58bd2c-6d10-4f8b-af18-4177b1d57416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:31.021458 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0ab45a-ae04-45ef-8171-b4f3dc66f120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:31.023232 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ab45a-ae04-45ef-8171-b4f3dc66f120 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:31.023707 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ab45a-ae04-45ef-8171-b4f3dc66f120 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:31.024162 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ab45a-ae04-45ef-8171-b4f3dc66f120 None admin] PATH_INFO: `/v3/users/6962122ffdbe4d6288f017e98d1e8ecb` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:31.025725 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0ab45a-ae04-45ef-8171-b4f3dc66f120 None admin] RBAC: Authorizing `identity:get_user(user_id=6962122ffdbe4d6288f017e98d1e8ecb)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:31.027449 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0ab45a-ae04-45ef-8171-b4f3dc66f120 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:31.029308 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 78/157] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 6 16:02:30 2022] GET /identity/v3/users/6962122ffdbe4d6288f017e98d1e8ecb => generated 275 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:31.030615 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0ab45a-ae04-45ef-8171-b4f3dc66f120 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:31.158218 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 80/158] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:02:31.159120 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a58bd2c-6d10-4f8b-af18-4177b1d57416 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:31.217695 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0ab45a-ae04-45ef-8171-b4f3dc66f120 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:31.225126 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:31.262124 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:31.287955 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9896fc51-e4b8-4bba-8105-5b87de643fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:31.289034 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9896fc51-e4b8-4bba-8105-5b87de643fa6 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:31.289545 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9896fc51-e4b8-4bba-8105-5b87de643fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:31.289997 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9896fc51-e4b8-4bba-8105-5b87de643fa6 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:31.290855 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9896fc51-e4b8-4bba-8105-5b87de643fa6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:31.291338 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9896fc51-e4b8-4bba-8105-5b87de643fa6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:31.292928 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9896fc51-e4b8-4bba-8105-5b87de643fa6 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:31.311184 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 79/159] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 6 16:02:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:31.321850 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9896fc51-e4b8-4bba-8105-5b87de643fa6 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:31.379987 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a58bd2c-6d10-4f8b-af18-4177b1d57416 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:31.429845 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f32b4f-8b97-41ef-bbed-d32d8a364cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:31.430450 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f32b4f-8b97-41ef-bbed-d32d8a364cdf None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:31.430733 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f32b4f-8b97-41ef-bbed-d32d8a364cdf None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:31.431048 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f32b4f-8b97-41ef-bbed-d32d8a364cdf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:31.526941 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f32b4f-8b97-41ef-bbed-d32d8a364cdf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:31.528445 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f32b4f-8b97-41ef-bbed-d32d8a364cdf None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:31.550887 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9896fc51-e4b8-4bba-8105-5b87de643fa6 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:31.570671 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:31.641485 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91446e61-49c4-43f8-8865-e4a32e13be4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:31.642146 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91446e61-49c4-43f8-8865-e4a32e13be4f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:31.642424 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91446e61-49c4-43f8-8865-e4a32e13be4f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:31.642693 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91446e61-49c4-43f8-8865-e4a32e13be4f None admin] PATH_INFO: `/v3/users/824c9880f41b4962b83243abf75b8162` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:31.643880 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91446e61-49c4-43f8-8865-e4a32e13be4f None admin] RBAC: Authorizing `identity:get_user(user_id=824c9880f41b4962b83243abf75b8162)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:31.645562 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91446e61-49c4-43f8-8865-e4a32e13be4f None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:31.648549 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 80/160] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 6 16:02:31 2022] GET /identity/v3/users/824c9880f41b4962b83243abf75b8162 => generated 247 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:31.653235 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91446e61-49c4-43f8-8865-e4a32e13be4f None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:31.658822 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 81/161] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 6 16:02:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 16:02:31.664781 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f32b4f-8b97-41ef-bbed-d32d8a364cdf None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:31.763049 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:31.788202 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f32b4f-8b97-41ef-bbed-d32d8a364cdf None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:31.818368 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:31.829240 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ff3bf6-4e1f-437e-a166-318f26b23eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:31.829882 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff3bf6-4e1f-437e-a166-318f26b23eae None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:31.830159 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff3bf6-4e1f-437e-a166-318f26b23eae None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:31.830427 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff3bf6-4e1f-437e-a166-318f26b23eae None admin] PATH_INFO: `/v3/projects/5e7a87d573a8446bbcbf9e28149a392f` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:31.831703 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ff3bf6-4e1f-437e-a166-318f26b23eae None admin] RBAC: Authorizing `identity:get_project(project_id=5e7a87d573a8446bbcbf9e28149a392f)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:31.833587 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ff3bf6-4e1f-437e-a166-318f26b23eae None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:31.842651 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 82/162] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 6 16:02:31 2022] GET /identity/v3/projects/5e7a87d573a8446bbcbf9e28149a392f => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:02:31.844950 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3689596c-dec9-45c6-91d0-90ca80441184 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:31.845443 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3689596c-dec9-45c6-91d0-90ca80441184 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:31.845717 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3689596c-dec9-45c6-91d0-90ca80441184 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:31.846969 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 83/163] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:02:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:02:31.852943 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3689596c-dec9-45c6-91d0-90ca80441184 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:31.869468 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91446e61-49c4-43f8-8865-e4a32e13be4f None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:31.910395 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f816d3bb-9aa8-47e2-ba55-12019f9f6d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:31.911025 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f816d3bb-9aa8-47e2-ba55-12019f9f6d70 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:31.911304 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f816d3bb-9aa8-47e2-ba55-12019f9f6d70 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:31.911579 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f816d3bb-9aa8-47e2-ba55-12019f9f6d70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:32.071809 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3689596c-dec9-45c6-91d0-90ca80441184 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:32.080202 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f816d3bb-9aa8-47e2-ba55-12019f9f6d70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:32.081767 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f816d3bb-9aa8-47e2-ba55-12019f9f6d70 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:32.146066 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf88c0b-4a73-4184-8438-21ba1cebf58c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:32.146714 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf88c0b-4a73-4184-8438-21ba1cebf58c None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:32.146988 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf88c0b-4a73-4184-8438-21ba1cebf58c None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:32.147256 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf88c0b-4a73-4184-8438-21ba1cebf58c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:32.176805 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf88c0b-4a73-4184-8438-21ba1cebf58c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:02:32.186848 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-0bf88c0b-4a73-4184-8438-21ba1cebf58c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 16:02:32.186848 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:32.186848 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:32.186848 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:32.186848 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:32.186848 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:32.186848 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:32.186848 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:32.186848 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:32.186848 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:32.186848 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:32.186848 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:32.186848 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:02:32.186848 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:02:32.186848 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:02:32.186848 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:02:32.189155 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:02:32.189155 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:02:32.189155 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:02:32.189155 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:02:32.189155 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:32.189155 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:32.189155 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:32.189155 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:32.189155 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:32.189155 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:32.189155 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:32.189155 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:32.189155 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:32.189155 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:32.189155 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:32.189155 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:32.191091 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:32.191091 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:32.191091 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:32.191091 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:32.191091 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:02:32.191091 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:02:32.191091 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:02:32.191091 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:02:32.191091 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:02:32.191091 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:02:32.191091 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 16:02:32.191091 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:02:32.191091 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 84/164] 10.222.0.43 () {56 vars in 1114 bytes} [Tue Sep 6 16:02:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:02:32.197963 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf88c0b-4a73-4184-8438-21ba1cebf58c None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:32.206209 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:32.310445 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 81/165] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 6 16:02:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 658 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 06 16:02:32.314788 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c07329d-a0bf-4e5c-9df5-d3d47c39fd10 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:32.317599 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c07329d-a0bf-4e5c-9df5-d3d47c39fd10 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:32.317894 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c07329d-a0bf-4e5c-9df5-d3d47c39fd10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:32.346503 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:32.361107 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-7c07329d-a0bf-4e5c-9df5-d3d47c39fd10 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:32.362312 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7c07329d-a0bf-4e5c-9df5-d3d47c39fd10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:32.448096 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 82/166] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:02:32.450688 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf88c0b-4a73-4184-8438-21ba1cebf58c None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:32.453118 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c07329d-a0bf-4e5c-9df5-d3d47c39fd10 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:32.521532 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283eaae6-2b85-497f-ba0f-9d6b86848b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:32.522167 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283eaae6-2b85-497f-ba0f-9d6b86848b11 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:32.522437 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283eaae6-2b85-497f-ba0f-9d6b86848b11 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:32.522724 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283eaae6-2b85-497f-ba0f-9d6b86848b11 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:32.523558 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283eaae6-2b85-497f-ba0f-9d6b86848b11 None admin] RBAC: Adding query filter params (role.id=15252b0b89894c1e8a6bd9a98b1c3ab7, scope.project.id=5e7a87d573a8446bbcbf9e28149a392f, user.id=6962122ffdbe4d6288f017e98d1e8ecb) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:32.523861 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283eaae6-2b85-497f-ba0f-9d6b86848b11 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:32.525389 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283eaae6-2b85-497f-ba0f-9d6b86848b11 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:32.546770 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 85/167] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 6 16:02:32 2022] GET /identity/v3/role_assignments?user.id=6962122ffdbe4d6288f017e98d1e8ecb&scope.project.id=5e7a87d573a8446bbcbf9e28149a392f&role.id=15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 608 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:32.547414 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283eaae6-2b85-497f-ba0f-9d6b86848b11 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:32.587430 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:32.599508 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c07329d-a0bf-4e5c-9df5-d3d47c39fd10 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:32.653132 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173f8568-8cf8-412c-aadd-9bbac7ed303e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:32.653824 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f8568-8cf8-412c-aadd-9bbac7ed303e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:32.654108 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f8568-8cf8-412c-aadd-9bbac7ed303e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:32.654385 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f8568-8cf8-412c-aadd-9bbac7ed303e None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:32.655081 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173f8568-8cf8-412c-aadd-9bbac7ed303e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:32.655385 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173f8568-8cf8-412c-aadd-9bbac7ed303e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:32.656833 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173f8568-8cf8-412c-aadd-9bbac7ed303e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:32.666496 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 83/168] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 6 16:02:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:32.672709 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173f8568-8cf8-412c-aadd-9bbac7ed303e None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:32.804526 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173f8568-8cf8-412c-aadd-9bbac7ed303e None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:32.839881 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50dded7-dc0b-4638-9627-018c0b288c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:32.840536 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50dded7-dc0b-4638-9627-018c0b288c74 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:32.840839 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50dded7-dc0b-4638-9627-018c0b288c74 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:32.841117 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50dded7-dc0b-4638-9627-018c0b288c74 None admin] PATH_INFO: `/v3/users/6962122ffdbe4d6288f017e98d1e8ecb` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:32.842295 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50dded7-dc0b-4638-9627-018c0b288c74 None admin] RBAC: Authorizing `identity:get_user(user_id=6962122ffdbe4d6288f017e98d1e8ecb)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:32.843772 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50dded7-dc0b-4638-9627-018c0b288c74 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:32.845541 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 84/169] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 6 16:02:32 2022] GET /identity/v3/users/6962122ffdbe4d6288f017e98d1e8ecb => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:32.849481 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283eaae6-2b85-497f-ba0f-9d6b86848b11 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:32.851370 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50dded7-dc0b-4638-9627-018c0b288c74 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:32.872118 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:32.922852 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:32.950015 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6a1ffc-b99c-41ee-b4b8-2eb1a86190c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:32.950666 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a1ffc-b99c-41ee-b4b8-2eb1a86190c9 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:32.950942 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a1ffc-b99c-41ee-b4b8-2eb1a86190c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:32.951210 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a1ffc-b99c-41ee-b4b8-2eb1a86190c9 None admin] PATH_INFO: `/v3/projects/5e7a87d573a8446bbcbf9e28149a392f` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:32.957838 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6a1ffc-b99c-41ee-b4b8-2eb1a86190c9 None admin] RBAC: Authorizing `identity:get_project(project_id=5e7a87d573a8446bbcbf9e28149a392f)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:32.959338 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6a1ffc-b99c-41ee-b4b8-2eb1a86190c9 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:32.961453 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 86/170] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 6 16:02:32 2022] GET /identity/v3/projects/5e7a87d573a8446bbcbf9e28149a392f => generated 298 bytes in 414 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:32.969711 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6a1ffc-b99c-41ee-b4b8-2eb1a86190c9 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:32.977043 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50dded7-dc0b-4638-9627-018c0b288c74 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:33.024298 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7470a7c0-e8a0-4815-be09-6b98cf95129a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:33.024986 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7470a7c0-e8a0-4815-be09-6b98cf95129a None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:33.025263 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7470a7c0-e8a0-4815-be09-6b98cf95129a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:33.025533 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7470a7c0-e8a0-4815-be09-6b98cf95129a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:33.123347 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7470a7c0-e8a0-4815-be09-6b98cf95129a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:33.124782 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7470a7c0-e8a0-4815-be09-6b98cf95129a None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:33.126886 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6a1ffc-b99c-41ee-b4b8-2eb1a86190c9 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:33.152612 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:33.247842 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 85/171] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 6 16:02:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 06 16:02:33.253398 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7470a7c0-e8a0-4815-be09-6b98cf95129a None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:33.267045 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c47891-b0e0-4cb2-a838-1efa4d69e8b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:33.267675 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c47891-b0e0-4cb2-a838-1efa4d69e8b6 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:33.267950 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c47891-b0e0-4cb2-a838-1efa4d69e8b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:33.268216 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c47891-b0e0-4cb2-a838-1efa4d69e8b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:33.269000 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c47891-b0e0-4cb2-a838-1efa4d69e8b6 None admin] RBAC: Adding query filter params (role.id=edaabdea8be84a7980fb2c651fe52f63, scope.project.id=5e7a87d573a8446bbcbf9e28149a392f, user.id=824c9880f41b4962b83243abf75b8162) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:33.269303 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c47891-b0e0-4cb2-a838-1efa4d69e8b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:33.270765 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c47891-b0e0-4cb2-a838-1efa4d69e8b6 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:33.291081 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 87/172] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 6 16:02:32 2022] GET /identity/v3/role_assignments?user.id=824c9880f41b4962b83243abf75b8162&scope.project.id=5e7a87d573a8446bbcbf9e28149a392f&role.id=edaabdea8be84a7980fb2c651fe52f63 => generated 608 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:02:33.292378 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ab83a-318f-48a8-b7ec-31ebaca65f44 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:33.292659 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ab83a-318f-48a8-b7ec-31ebaca65f44 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:33.292951 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ab83a-318f-48a8-b7ec-31ebaca65f44 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:33.294399 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 88/173] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:02:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:02:33.295102 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847ab83a-318f-48a8-b7ec-31ebaca65f44 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:33.362198 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7470a7c0-e8a0-4815-be09-6b98cf95129a None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:33.381241 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:33.401216 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a960729c-05db-42aa-9241-cfbf06a9c9a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:33.401992 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a960729c-05db-42aa-9241-cfbf06a9c9a6 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:33.402274 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a960729c-05db-42aa-9241-cfbf06a9c9a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:33.402551 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a960729c-05db-42aa-9241-cfbf06a9c9a6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:33.414558 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a960729c-05db-42aa-9241-cfbf06a9c9a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:02:33.419835 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-a960729c-05db-42aa-9241-cfbf06a9c9a6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 16:02:33.419835 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:33.419835 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:33.419835 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:33.419835 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:33.419835 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:33.419835 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:33.419835 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:33.419835 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:33.419835 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:33.419835 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:33.419835 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:33.419835 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:02:33.419835 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:02:33.419835 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:02:33.419835 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:02:33.420785 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:02:33.420785 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:02:33.420785 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:02:33.420785 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:02:33.420785 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:33.420785 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:33.420785 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:33.420785 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:33.420785 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:33.420785 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:33.420785 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:33.420785 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:33.420785 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:33.420785 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:33.420785 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:33.420785 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:33.421600 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:33.421600 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:33.421600 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:33.421600 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:33.421600 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:02:33.421600 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:02:33.421600 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:02:33.421600 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:02:33.421600 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:02:33.421600 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:02:33.421600 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 16:02:33.421600 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:02:33.425863 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 86/174] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 6 16:02:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:02:33.427517 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2bb923-eb9a-4190-b385-6652593c120a None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:33.427805 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2bb923-eb9a-4190-b385-6652593c120a None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:33.428083 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2bb923-eb9a-4190-b385-6652593c120a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:33.454876 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-ef2bb923-eb9a-4190-b385-6652593c120a None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:33.456475 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ef2bb923-eb9a-4190-b385-6652593c120a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:33.510978 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:33.533171 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 87/175] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:02:33.534195 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2bb923-eb9a-4190-b385-6652593c120a None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:33.605267 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847ab83a-318f-48a8-b7ec-31ebaca65f44 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:33.644965 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f11fba-46d8-4078-ab05-ea6b120c7f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:33.645523 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f11fba-46d8-4078-ab05-ea6b120c7f96 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:33.645845 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f11fba-46d8-4078-ab05-ea6b120c7f96 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:33.646156 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f11fba-46d8-4078-ab05-ea6b120c7f96 None admin] PATH_INFO: `/v3/projects/5e7a87d573a8446bbcbf9e28149a392f` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:33.648076 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f11fba-46d8-4078-ab05-ea6b120c7f96 None admin] RBAC: Authorizing `identity:get_project(project_id=5e7a87d573a8446bbcbf9e28149a392f)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:33.649631 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f11fba-46d8-4078-ab05-ea6b120c7f96 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:33.651559 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 89/176] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 6 16:02:33 2022] GET /identity/v3/projects/5e7a87d573a8446bbcbf9e28149a392f => generated 298 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:33.652596 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f7e40f-a08f-4af9-9ee1-316d20433730 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:33.652950 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f7e40f-a08f-4af9-9ee1-316d20433730 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:33.653219 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f7e40f-a08f-4af9-9ee1-316d20433730 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:33.654408 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 90/177] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:02:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:02:33.657591 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2bb923-eb9a-4190-b385-6652593c120a None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:33.659934 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7626ef36-679a-4917-bab4-9c944efa596e None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:33.660284 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7626ef36-679a-4917-bab4-9c944efa596e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:33.660642 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7626ef36-679a-4917-bab4-9c944efa596e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:33.670908 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:33.684551 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-7626ef36-679a-4917-bab4-9c944efa596e None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:33.685888 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-7626ef36-679a-4917-bab4-9c944efa596e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:33.704436 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d811d89b-74e1-4d79-877c-5e737bb25a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:33.705202 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d811d89b-74e1-4d79-877c-5e737bb25a7b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:33.705482 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d811d89b-74e1-4d79-877c-5e737bb25a7b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:33.705848 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d811d89b-74e1-4d79-877c-5e737bb25a7b None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:33.706676 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d811d89b-74e1-4d79-877c-5e737bb25a7b None admin] RBAC: Adding query filter params (name=member) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:33.707392 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d811d89b-74e1-4d79-877c-5e737bb25a7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:33.708954 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d811d89b-74e1-4d79-877c-5e737bb25a7b None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:33.718797 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 88/178] 10.222.0.43 () {56 vars in 1101 bytes} [Tue Sep 6 16:02:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:33.719825 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d811d89b-74e1-4d79-877c-5e737bb25a7b None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:33.761175 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 91/179] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:02:33.763464 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23266393-3ffe-4fae-99d9-79769a22c574 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:33.763750 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23266393-3ffe-4fae-99d9-79769a22c574 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:33.764018 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23266393-3ffe-4fae-99d9-79769a22c574 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:33.765348 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 92/180] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:02:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:02:33.766413 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a1ea49-a93f-4d55-94f6-1aad994657af None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:33.766685 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a1ea49-a93f-4d55-94f6-1aad994657af None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:33.766956 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a1ea49-a93f-4d55-94f6-1aad994657af None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:33.768261 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 93/181] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:02:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:02:33.769034 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a1ea49-a93f-4d55-94f6-1aad994657af None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:33.826481 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d811d89b-74e1-4d79-877c-5e737bb25a7b None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:33.830995 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:33.863188 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe433fb8-6aaf-43f3-8d4a-ad44c5ef215a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:33.864168 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe433fb8-6aaf-43f3-8d4a-ad44c5ef215a None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:33.864671 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe433fb8-6aaf-43f3-8d4a-ad44c5ef215a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:33.865097 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe433fb8-6aaf-43f3-8d4a-ad44c5ef215a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:33.865969 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe433fb8-6aaf-43f3-8d4a-ad44c5ef215a None admin] RBAC: Adding query filter params (role.id=61df19b017494bdcadc9a5e9d8bb9e52, scope.project.id=5e7a87d573a8446bbcbf9e28149a392f, user.id=6962122ffdbe4d6288f017e98d1e8ecb) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:33.866559 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe433fb8-6aaf-43f3-8d4a-ad44c5ef215a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:33.868324 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe433fb8-6aaf-43f3-8d4a-ad44c5ef215a None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:33.874886 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 89/182] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 6 16:02:33 2022] GET /identity/v3/role_assignments?user.id=6962122ffdbe4d6288f017e98d1e8ecb&scope.project.id=5e7a87d573a8446bbcbf9e28149a392f&role.id=61df19b017494bdcadc9a5e9d8bb9e52 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:33.876301 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdca372e-e3b5-4c71-b28d-f630f917708b None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:33.876810 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdca372e-e3b5-4c71-b28d-f630f917708b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:33.877279 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdca372e-e3b5-4c71-b28d-f630f917708b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:33.886037 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a1ea49-a93f-4d55-94f6-1aad994657af None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:33.898210 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-bdca372e-e3b5-4c71-b28d-f630f917708b None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:33.899832 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-bdca372e-e3b5-4c71-b28d-f630f917708b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:33.927745 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce9980d-8dbc-4246-a0a7-5739be98e467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:33.928515 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce9980d-8dbc-4246-a0a7-5739be98e467 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:33.928993 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce9980d-8dbc-4246-a0a7-5739be98e467 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:33.929301 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce9980d-8dbc-4246-a0a7-5739be98e467 None admin] PATH_INFO: `/v3/users/6962122ffdbe4d6288f017e98d1e8ecb` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:33.930629 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce9980d-8dbc-4246-a0a7-5739be98e467 None admin] RBAC: Authorizing `identity:get_user(user_id=6962122ffdbe4d6288f017e98d1e8ecb)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:33.931798 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:33.934237 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce9980d-8dbc-4246-a0a7-5739be98e467 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:33.936600 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 94/183] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 6 16:02:33 2022] GET /identity/v3/users/6962122ffdbe4d6288f017e98d1e8ecb => generated 275 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:33.938139 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46440283-34b8-4167-b6a2-191463726f56 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:33.938967 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46440283-34b8-4167-b6a2-191463726f56 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:33.939308 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46440283-34b8-4167-b6a2-191463726f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:33.962146 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-46440283-34b8-4167-b6a2-191463726f56 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:33.963458 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-46440283-34b8-4167-b6a2-191463726f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:33.980957 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 90/184] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:02:33.982123 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdca372e-e3b5-4c71-b28d-f630f917708b None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:34.061694 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 95/185] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:02:34.062596 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46440283-34b8-4167-b6a2-191463726f56 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:34.102655 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdca372e-e3b5-4c71-b28d-f630f917708b None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:34.124838 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:34.142149 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e6b22a-2f15-46ec-80c4-87bef6db38d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:34.142796 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e6b22a-2f15-46ec-80c4-87bef6db38d7 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:34.143116 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e6b22a-2f15-46ec-80c4-87bef6db38d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:34.143343 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e6b22a-2f15-46ec-80c4-87bef6db38d7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:34.154582 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e6b22a-2f15-46ec-80c4-87bef6db38d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:02:34.160281 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-d1e6b22a-2f15-46ec-80c4-87bef6db38d7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:02:34.160281 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:34.160281 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:34.160281 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:34.160281 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:34.160281 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:34.160281 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:34.160281 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:34.160281 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:34.160281 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:34.160281 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:34.160281 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:34.160281 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:02:34.160281 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:02:34.160281 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:02:34.160281 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:02:34.161241 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:02:34.161241 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:02:34.161241 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:02:34.161241 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:02:34.161241 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:34.161241 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:34.161241 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:34.161241 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:34.161241 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:34.161241 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:34.161241 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:34.161241 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:34.161241 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:34.161241 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:34.161241 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:34.161241 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:34.162055 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:34.162055 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:34.162055 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:34.162055 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:34.162055 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:02:34.162055 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:02:34.162055 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:02:34.162055 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:02:34.162055 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:02:34.162055 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:02:34.162055 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:02:34.162055 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:02:34.162055 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 91/186] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 6 16:02:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:02:34.162740 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e6b22a-2f15-46ec-80c4-87bef6db38d7 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:34.190803 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46440283-34b8-4167-b6a2-191463726f56 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:34.193151 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:34.228442 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a417dd4-196b-4480-84b7-d45be72e59b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:34.229128 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a417dd4-196b-4480-84b7-d45be72e59b8 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:34.229455 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a417dd4-196b-4480-84b7-d45be72e59b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:34.229727 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a417dd4-196b-4480-84b7-d45be72e59b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:34.277368 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e6b22a-2f15-46ec-80c4-87bef6db38d7 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:34.303129 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a417dd4-196b-4480-84b7-d45be72e59b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:34.304666 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a417dd4-196b-4480-84b7-d45be72e59b8 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:34.315255 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b36e05-ec28-4b0f-a0b5-966dbab73a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:34.315960 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b36e05-ec28-4b0f-a0b5-966dbab73a41 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:34.316242 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b36e05-ec28-4b0f-a0b5-966dbab73a41 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:34.316613 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b36e05-ec28-4b0f-a0b5-966dbab73a41 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:34.317783 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b36e05-ec28-4b0f-a0b5-966dbab73a41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:34.319332 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b36e05-ec28-4b0f-a0b5-966dbab73a41 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:34.321473 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 92/187] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:02:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:34.327397 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b36e05-ec28-4b0f-a0b5-966dbab73a41 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:34.360245 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:34.410856 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 96/188] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 6 16:02:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 16:02:34.411613 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a417dd4-196b-4480-84b7-d45be72e59b8 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:34.429175 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:34.442991 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b36e05-ec28-4b0f-a0b5-966dbab73a41 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:34.478958 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3942f79-3aba-451d-a030-7e3169236ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:34.479571 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3942f79-3aba-451d-a030-7e3169236ce9 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:34.479844 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3942f79-3aba-451d-a030-7e3169236ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:34.480107 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3942f79-3aba-451d-a030-7e3169236ce9 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:34.480931 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3942f79-3aba-451d-a030-7e3169236ce9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:34.481230 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3942f79-3aba-451d-a030-7e3169236ce9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:34.482622 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3942f79-3aba-451d-a030-7e3169236ce9 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:34.490901 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 93/189] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 6 16:02:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:34.491747 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3942f79-3aba-451d-a030-7e3169236ce9 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:34.513634 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a417dd4-196b-4480-84b7-d45be72e59b8 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:34.546172 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6e2214-1141-4bbd-b178-3a8cfd33695d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:34.546893 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e2214-1141-4bbd-b178-3a8cfd33695d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:34.547170 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e2214-1141-4bbd-b178-3a8cfd33695d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:34.547441 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e2214-1141-4bbd-b178-3a8cfd33695d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:34.557404 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6e2214-1141-4bbd-b178-3a8cfd33695d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:02:34.562241 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-bf6e2214-1141-4bbd-b178-3a8cfd33695d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 16:02:34.562241 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:34.562241 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:34.562241 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:34.562241 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:34.562241 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:34.562241 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:34.562241 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:34.562241 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:34.562241 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:34.562241 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:34.562241 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:34.562241 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:02:34.562241 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:02:34.562241 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:02:34.562241 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:02:34.563282 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:02:34.563282 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:02:34.563282 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:02:34.563282 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:02:34.563282 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:34.563282 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:34.563282 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:34.563282 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:34.563282 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:34.563282 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:34.563282 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:34.563282 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:34.563282 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:34.563282 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:34.563282 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:34.563282 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:34.565559 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:34.565559 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:34.565559 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:34.565559 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:34.565559 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:02:34.565559 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:02:34.565559 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:02:34.565559 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:02:34.565559 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:02:34.565559 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:02:34.565559 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 16:02:34.565559 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:02:34.565559 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 97/190] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 6 16:02:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:02:34.565559 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6e2214-1141-4bbd-b178-3a8cfd33695d None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:34.578133 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:34.591318 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3942f79-3aba-451d-a030-7e3169236ce9 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:34.624294 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864b0738-0be5-4dba-b726-6f395ebb5f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:34.624938 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864b0738-0be5-4dba-b726-6f395ebb5f8c None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:34.625216 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864b0738-0be5-4dba-b726-6f395ebb5f8c None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:34.625488 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864b0738-0be5-4dba-b726-6f395ebb5f8c None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:34.626138 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864b0738-0be5-4dba-b726-6f395ebb5f8c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:34.627607 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864b0738-0be5-4dba-b726-6f395ebb5f8c None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:34.648671 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 94/191] 10.222.0.43 () {60 vars in 1132 bytes} [Tue Sep 6 16:02:34 2022] POST /identity/v3/users => generated 297 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:02:34.649512 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864b0738-0be5-4dba-b726-6f395ebb5f8c None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:34.662606 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:34.681015 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6e2214-1141-4bbd-b178-3a8cfd33695d None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:34.723004 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8330da2-a05d-4afa-bae9-849843da7b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:34.723965 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8330da2-a05d-4afa-bae9-849843da7b06 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:34.724451 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8330da2-a05d-4afa-bae9-849843da7b06 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:34.724721 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8330da2-a05d-4afa-bae9-849843da7b06 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:34.737066 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8330da2-a05d-4afa-bae9-849843da7b06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:02:34.742416 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-e8330da2-a05d-4afa-bae9-849843da7b06 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 16:02:34.742416 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:34.742416 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:34.742416 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:34.742416 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:34.742416 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:34.742416 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:34.742416 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:34.742416 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:34.742416 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:34.742416 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:34.742416 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:34.742416 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:02:34.742416 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:02:34.742416 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:02:34.742416 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:02:34.743365 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:02:34.743365 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:02:34.743365 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:02:34.743365 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:02:34.743365 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:34.743365 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:34.743365 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:34.743365 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:34.743365 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:34.743365 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:34.743365 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:34.743365 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:34.743365 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:34.743365 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:34.743365 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:34.743365 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:34.744163 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:34.744163 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:34.744163 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:34.744163 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:34.744163 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:02:34.744163 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:02:34.744163 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:02:34.744163 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:02:34.744163 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:02:34.744163 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:02:34.744163 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 16:02:34.744163 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:02:34.744163 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 98/192] 10.222.0.43 () {56 vars in 1114 bytes} [Tue Sep 6 16:02:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:02:34.744879 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8330da2-a05d-4afa-bae9-849843da7b06 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:34.773320 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864b0738-0be5-4dba-b726-6f395ebb5f8c None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:34.811396 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5e40b8-1a48-4f44-9d8d-5c1f4ea4a7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:34.812049 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5e40b8-1a48-4f44-9d8d-5c1f4ea4a7b8 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:34.812338 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5e40b8-1a48-4f44-9d8d-5c1f4ea4a7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:34.812596 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5e40b8-1a48-4f44-9d8d-5c1f4ea4a7b8 None admin] PATH_INFO: `/v3/projects/cfd6384b3080463397c5b619b3626b70` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:34.813763 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5e40b8-1a48-4f44-9d8d-5c1f4ea4a7b8 None admin] RBAC: Authorizing `identity:get_project(project_id=cfd6384b3080463397c5b619b3626b70)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:34.815314 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5e40b8-1a48-4f44-9d8d-5c1f4ea4a7b8 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:34.816997 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 95/193] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 6 16:02:34 2022] GET /identity/v3/projects/cfd6384b3080463397c5b619b3626b70 => generated 312 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:34.817835 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5e40b8-1a48-4f44-9d8d-5c1f4ea4a7b8 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:34.826432 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:34.873004 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8330da2-a05d-4afa-bae9-849843da7b06 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:34.906942 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:34.916033 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b04b6a8-9028-42c4-83d7-ed9c8c2344e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:34.917006 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b04b6a8-9028-42c4-83d7-ed9c8c2344e0 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:34.917285 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b04b6a8-9028-42c4-83d7-ed9c8c2344e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:34.917555 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b04b6a8-9028-42c4-83d7-ed9c8c2344e0 None admin] PATH_INFO: `/v3/users/a73439da3d3a43ca95973b14c17f910f` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:34.942774 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b04b6a8-9028-42c4-83d7-ed9c8c2344e0 None admin] RBAC: Authorizing `identity:get_user(user_id=a73439da3d3a43ca95973b14c17f910f)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:34.943283 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5e40b8-1a48-4f44-9d8d-5c1f4ea4a7b8 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:34.944328 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b04b6a8-9028-42c4-83d7-ed9c8c2344e0 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:34.946305 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 99/194] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 6 16:02:34 2022] GET /identity/v3/users/a73439da3d3a43ca95973b14c17f910f => generated 283 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:34.946913 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b04b6a8-9028-42c4-83d7-ed9c8c2344e0 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:34.985177 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d344cb-6603-40d8-85d1-ef219f43acbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:34.985797 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d344cb-6603-40d8-85d1-ef219f43acbe None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:34.986073 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d344cb-6603-40d8-85d1-ef219f43acbe None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:34.986342 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d344cb-6603-40d8-85d1-ef219f43acbe None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:34.987010 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d344cb-6603-40d8-85d1-ef219f43acbe None admin] RBAC: Adding query filter params (name=reader) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:34.987309 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d344cb-6603-40d8-85d1-ef219f43acbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:34.988693 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d344cb-6603-40d8-85d1-ef219f43acbe None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:34.999817 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 96/195] 10.222.0.43 () {56 vars in 1101 bytes} [Tue Sep 6 16:02:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:35.005083 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d344cb-6603-40d8-85d1-ef219f43acbe None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:35.070552 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b04b6a8-9028-42c4-83d7-ed9c8c2344e0 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:35.107773 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:35.115749 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d344cb-6603-40d8-85d1-ef219f43acbe None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:35.118986 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5f5e10-da82-47e2-9efa-4f78bfd73352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:35.119866 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5f5e10-da82-47e2-9efa-4f78bfd73352 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:35.120352 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5f5e10-da82-47e2-9efa-4f78bfd73352 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:35.120856 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5f5e10-da82-47e2-9efa-4f78bfd73352 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:35.121723 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5f5e10-da82-47e2-9efa-4f78bfd73352 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:35.122212 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5f5e10-da82-47e2-9efa-4f78bfd73352 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:35.124125 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5f5e10-da82-47e2-9efa-4f78bfd73352 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:35.136551 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 100/196] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 6 16:02:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:35.137845 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5f5e10-da82-47e2-9efa-4f78bfd73352 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:35.157530 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc1fb59-3115-4013-8ff0-92fca3720b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:35.158225 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc1fb59-3115-4013-8ff0-92fca3720b78 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:35.158422 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc1fb59-3115-4013-8ff0-92fca3720b78 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:35.158697 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc1fb59-3115-4013-8ff0-92fca3720b78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:35.159710 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc1fb59-3115-4013-8ff0-92fca3720b78 None admin] RBAC: Adding query filter params (role.id=15252b0b89894c1e8a6bd9a98b1c3ab7, scope.project.id=cfd6384b3080463397c5b619b3626b70, user.id=6962122ffdbe4d6288f017e98d1e8ecb) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:35.160013 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc1fb59-3115-4013-8ff0-92fca3720b78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:35.161657 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc1fb59-3115-4013-8ff0-92fca3720b78 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:35.169062 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 97/197] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 6 16:02:35 2022] GET /identity/v3/role_assignments?user.id=6962122ffdbe4d6288f017e98d1e8ecb&scope.project.id=cfd6384b3080463397c5b619b3626b70&role.id=15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:35.176931 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc1fb59-3115-4013-8ff0-92fca3720b78 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:35.185043 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:35.329697 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc1fb59-3115-4013-8ff0-92fca3720b78 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:35.372260 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70a95b3-35ce-4244-9216-af5c927e1382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:35.372634 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5f5e10-da82-47e2-9efa-4f78bfd73352 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:35.373084 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70a95b3-35ce-4244-9216-af5c927e1382 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:35.373268 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70a95b3-35ce-4244-9216-af5c927e1382 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:35.373472 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70a95b3-35ce-4244-9216-af5c927e1382 None admin] PATH_INFO: `/v3/users/4d3613f1e41b4d6c8dd2e2b72002ad79` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:35.374772 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70a95b3-35ce-4244-9216-af5c927e1382 None admin] RBAC: Authorizing `identity:get_user(user_id=4d3613f1e41b4d6c8dd2e2b72002ad79)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:35.377177 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70a95b3-35ce-4244-9216-af5c927e1382 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:35.378859 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 98/198] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 6 16:02:35 2022] GET /identity/v3/users/4d3613f1e41b4d6c8dd2e2b72002ad79 => generated 289 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:35.379629 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70a95b3-35ce-4244-9216-af5c927e1382 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:35.427300 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea4f25d-8c10-4ed8-a445-7535d1a78b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:35.427838 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea4f25d-8c10-4ed8-a445-7535d1a78b67 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:35.428120 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea4f25d-8c10-4ed8-a445-7535d1a78b67 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:35.428444 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea4f25d-8c10-4ed8-a445-7535d1a78b67 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:35.472018 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:35.494336 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:35.506993 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea4f25d-8c10-4ed8-a445-7535d1a78b67 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:35.508537 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea4f25d-8c10-4ed8-a445-7535d1a78b67 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:35.514788 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70a95b3-35ce-4244-9216-af5c927e1382 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:35.560447 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29986792-10f1-4d57-b05c-4cf9c7809bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:35.561639 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29986792-10f1-4d57-b05c-4cf9c7809bfc None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:35.561919 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29986792-10f1-4d57-b05c-4cf9c7809bfc None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:35.562188 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29986792-10f1-4d57-b05c-4cf9c7809bfc None admin] PATH_INFO: `/v3/users/a73439da3d3a43ca95973b14c17f910f` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:35.563753 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29986792-10f1-4d57-b05c-4cf9c7809bfc None admin] RBAC: Authorizing `identity:get_user(user_id=a73439da3d3a43ca95973b14c17f910f)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:35.565783 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29986792-10f1-4d57-b05c-4cf9c7809bfc None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:35.568153 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 99/199] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 6 16:02:35 2022] GET /identity/v3/users/a73439da3d3a43ca95973b14c17f910f => generated 283 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:35.569292 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29986792-10f1-4d57-b05c-4cf9c7809bfc None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:35.640173 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 101/200] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 6 16:02:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 16:02:35.641080 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea4f25d-8c10-4ed8-a445-7535d1a78b67 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:35.700347 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29986792-10f1-4d57-b05c-4cf9c7809bfc None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:35.739406 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96cc74e-4ad4-44a7-b060-4d60da08b16f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:35.740277 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96cc74e-4ad4-44a7-b060-4d60da08b16f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:35.740563 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96cc74e-4ad4-44a7-b060-4d60da08b16f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:35.741120 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96cc74e-4ad4-44a7-b060-4d60da08b16f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:35.747652 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:35.756081 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:35.757776 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea4f25d-8c10-4ed8-a445-7535d1a78b67 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:35.795737 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcb20b4-3e1a-41c8-b9c3-ce52ab14ba9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:35.796358 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcb20b4-3e1a-41c8-b9c3-ce52ab14ba9d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:35.796694 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcb20b4-3e1a-41c8-b9c3-ce52ab14ba9d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:35.797008 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcb20b4-3e1a-41c8-b9c3-ce52ab14ba9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:35.843340 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96cc74e-4ad4-44a7-b060-4d60da08b16f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:35.844956 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96cc74e-4ad4-44a7-b060-4d60da08b16f None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:35.873923 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcb20b4-3e1a-41c8-b9c3-ce52ab14ba9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:35.875584 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcb20b4-3e1a-41c8-b9c3-ce52ab14ba9d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:35.964438 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 100/201] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 6 16:02:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 16:02:35.969685 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96cc74e-4ad4-44a7-b060-4d60da08b16f None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:35.981161 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:35.995861 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 102/202] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 6 16:02:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 16:02:35.997061 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcb20b4-3e1a-41c8-b9c3-ce52ab14ba9d None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:36.018576 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:36.098924 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96cc74e-4ad4-44a7-b060-4d60da08b16f None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:36.118314 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcb20b4-3e1a-41c8-b9c3-ce52ab14ba9d None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:36.149326 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74b5ed8-b985-4dbf-af0f-60e66469c900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:36.150931 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74b5ed8-b985-4dbf-af0f-60e66469c900 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:36.151368 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74b5ed8-b985-4dbf-af0f-60e66469c900 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:36.151777 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74b5ed8-b985-4dbf-af0f-60e66469c900 None admin] PATH_INFO: `/v3/projects/cf9744296b5341f08017259b44d86f94` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:36.153107 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74b5ed8-b985-4dbf-af0f-60e66469c900 None admin] RBAC: Authorizing `identity:get_project(project_id=cf9744296b5341f08017259b44d86f94)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:36.154700 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74b5ed8-b985-4dbf-af0f-60e66469c900 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:36.154836 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf6f423-8e06-4058-9db4-d0c9f502ef06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:36.155402 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf6f423-8e06-4058-9db4-d0c9f502ef06 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:36.155681 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf6f423-8e06-4058-9db4-d0c9f502ef06 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:36.155955 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf6f423-8e06-4058-9db4-d0c9f502ef06 None admin] PATH_INFO: `/v3/projects/5e7a87d573a8446bbcbf9e28149a392f` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:36.157126 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 101/203] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 6 16:02:35 2022] GET /identity/v3/projects/cf9744296b5341f08017259b44d86f94 => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:36.157452 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf6f423-8e06-4058-9db4-d0c9f502ef06 None admin] RBAC: Authorizing `identity:get_project(project_id=5e7a87d573a8446bbcbf9e28149a392f)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:36.158189 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74b5ed8-b985-4dbf-af0f-60e66469c900 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:36.159186 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf6f423-8e06-4058-9db4-d0c9f502ef06 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:36.161082 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 103/204] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 6 16:02:35 2022] GET /identity/v3/projects/5e7a87d573a8446bbcbf9e28149a392f => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:36.167103 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf6f423-8e06-4058-9db4-d0c9f502ef06 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:36.247567 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:36.283590 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74b5ed8-b985-4dbf-af0f-60e66469c900 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:36.293079 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:36.311108 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf6f423-8e06-4058-9db4-d0c9f502ef06 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:36.321772 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bcb89f-315c-4721-b042-9d02146ce83a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:36.322417 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bcb89f-315c-4721-b042-9d02146ce83a None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:36.322695 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bcb89f-315c-4721-b042-9d02146ce83a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:36.322967 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bcb89f-315c-4721-b042-9d02146ce83a None admin] PATH_INFO: `/v3/projects/cf9744296b5341f08017259b44d86f94` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:36.324108 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bcb89f-315c-4721-b042-9d02146ce83a None admin] RBAC: Authorizing `identity:get_project(project_id=cf9744296b5341f08017259b44d86f94)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:36.325700 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bcb89f-315c-4721-b042-9d02146ce83a None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:36.332775 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 102/205] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 6 16:02:36 2022] GET /identity/v3/projects/cf9744296b5341f08017259b44d86f94 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:36.332775 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bcb89f-315c-4721-b042-9d02146ce83a None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:36.364774 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f390f1b-7043-481c-bd15-d627172f7d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:36.365997 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f390f1b-7043-481c-bd15-d627172f7d94 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:36.366563 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f390f1b-7043-481c-bd15-d627172f7d94 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:36.367040 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f390f1b-7043-481c-bd15-d627172f7d94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:36.368953 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f390f1b-7043-481c-bd15-d627172f7d94 None admin] RBAC: Adding query filter params (role.id=edaabdea8be84a7980fb2c651fe52f63, scope.project.id=cf9744296b5341f08017259b44d86f94, user.id=a73439da3d3a43ca95973b14c17f910f) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:36.369289 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f390f1b-7043-481c-bd15-d627172f7d94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:36.370765 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f390f1b-7043-481c-bd15-d627172f7d94 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:36.382527 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 104/206] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 6 16:02:36 2022] GET /identity/v3/role_assignments?user.id=a73439da3d3a43ca95973b14c17f910f&scope.project.id=cf9744296b5341f08017259b44d86f94&role.id=edaabdea8be84a7980fb2c651fe52f63 => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:36.384702 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f390f1b-7043-481c-bd15-d627172f7d94 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:36.509225 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bcb89f-315c-4721-b042-9d02146ce83a None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:36.554847 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72cdcb6-c893-4c3b-8940-86bfde4f3f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:36.555949 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72cdcb6-c893-4c3b-8940-86bfde4f3f6c None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:36.556230 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72cdcb6-c893-4c3b-8940-86bfde4f3f6c None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:36.556503 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72cdcb6-c893-4c3b-8940-86bfde4f3f6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:36.557346 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72cdcb6-c893-4c3b-8940-86bfde4f3f6c None admin] RBAC: Adding query filter params (role.id=4908d6a02c4049e88249e9afbf7e2e9f, scope.project.id=5e7a87d573a8446bbcbf9e28149a392f, user.id=4d3613f1e41b4d6c8dd2e2b72002ad79) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:36.557658 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72cdcb6-c893-4c3b-8940-86bfde4f3f6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:36.559135 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72cdcb6-c893-4c3b-8940-86bfde4f3f6c None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:36.570746 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 103/207] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 6 16:02:36 2022] GET /identity/v3/role_assignments?user.id=4d3613f1e41b4d6c8dd2e2b72002ad79&scope.project.id=5e7a87d573a8446bbcbf9e28149a392f&role.id=4908d6a02c4049e88249e9afbf7e2e9f => generated 608 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:36.629249 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f390f1b-7043-481c-bd15-d627172f7d94 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:36.672666 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:36.690927 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb47a47-5642-4834-ad67-2a4a74465046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:36.691568 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb47a47-5642-4834-ad67-2a4a74465046 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:36.691849 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb47a47-5642-4834-ad67-2a4a74465046 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:36.692122 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb47a47-5642-4834-ad67-2a4a74465046 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:36.692912 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb47a47-5642-4834-ad67-2a4a74465046 None admin] RBAC: Adding query filter params (role.id=61df19b017494bdcadc9a5e9d8bb9e52, scope.project.id=cf9744296b5341f08017259b44d86f94, user.id=a73439da3d3a43ca95973b14c17f910f) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:36.693223 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb47a47-5642-4834-ad67-2a4a74465046 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:36.694829 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb47a47-5642-4834-ad67-2a4a74465046 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:36.705523 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 105/208] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 6 16:02:36 2022] GET /identity/v3/role_assignments?user.id=a73439da3d3a43ca95973b14c17f910f&scope.project.id=cf9744296b5341f08017259b44d86f94&role.id=61df19b017494bdcadc9a5e9d8bb9e52 => generated 259 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:36.826219 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719aae03-c1a9-406c-b267-c81f80ba2198 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:36.826606 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719aae03-c1a9-406c-b267-c81f80ba2198 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:36.826774 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719aae03-c1a9-406c-b267-c81f80ba2198 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:36.828456 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 104/209] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:02:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:02:36.834673 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7419e8-e219-4ce9-abee-8733a7be5afc None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:36.835058 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7419e8-e219-4ce9-abee-8733a7be5afc None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:36.835315 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7419e8-e219-4ce9-abee-8733a7be5afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:36.856056 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-0d7419e8-e219-4ce9-abee-8733a7be5afc None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:36.858887 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0d7419e8-e219-4ce9-abee-8733a7be5afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:36.922369 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ed028-64ea-4f2a-b2a8-b09960a4b9ae None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:36.922722 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ed028-64ea-4f2a-b2a8-b09960a4b9ae None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:36.922916 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ed028-64ea-4f2a-b2a8-b09960a4b9ae None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:36.924300 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 105/210] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:02:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 16:02:36.929866 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d674f17-1844-4e57-aaa8-ed6fd196df96 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:36.930139 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d674f17-1844-4e57-aaa8-ed6fd196df96 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:36.930413 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d674f17-1844-4e57-aaa8-ed6fd196df96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:36.943046 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 106/211] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:02:36.959193 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-2d674f17-1844-4e57-aaa8-ed6fd196df96 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:36.960955 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-2d674f17-1844-4e57-aaa8-ed6fd196df96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:36.992929 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:37.030496 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 106/212] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 16:02:37.224141 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7419e8-e219-4ce9-abee-8733a7be5afc None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:37.297669 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d674f17-1844-4e57-aaa8-ed6fd196df96 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:37.327857 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7419e8-e219-4ce9-abee-8733a7be5afc None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:37.345968 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:37.369751 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf05c7b-5681-4e7c-91ef-2287c1d63f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:37.370397 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf05c7b-5681-4e7c-91ef-2287c1d63f7f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:37.370676 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf05c7b-5681-4e7c-91ef-2287c1d63f7f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:37.370931 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf05c7b-5681-4e7c-91ef-2287c1d63f7f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:37.382140 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf05c7b-5681-4e7c-91ef-2287c1d63f7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:02:37.387230 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-faf05c7b-5681-4e7c-91ef-2287c1d63f7f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 16:02:37.387230 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:37.387230 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:37.387230 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:37.387230 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:37.387230 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:37.387230 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:37.387230 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:37.387230 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:37.387230 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:37.387230 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:37.387230 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:37.387230 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:02:37.387230 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:02:37.387230 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:02:37.387230 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:02:37.388498 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:02:37.388498 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:02:37.388498 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:02:37.388498 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:02:37.388498 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:37.388498 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:37.388498 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:37.388498 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:37.388498 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:37.388498 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:37.388498 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:37.388498 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:37.388498 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:37.388498 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:37.388498 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:37.388498 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:37.389318 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:37.389318 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:37.389318 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:37.389318 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:37.389318 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:02:37.389318 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:02:37.389318 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:02:37.389318 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:02:37.389318 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:02:37.389318 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:02:37.389318 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 16:02:37.389318 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:02:37.389318 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 107/213] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 6 16:02:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:02:37.398128 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf05c7b-5681-4e7c-91ef-2287c1d63f7f None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:37.408632 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d674f17-1844-4e57-aaa8-ed6fd196df96 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:37.443096 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f37bb4-c298-41c6-a061-634eb2509baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:37.443742 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f37bb4-c298-41c6-a061-634eb2509baf None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:37.444021 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f37bb4-c298-41c6-a061-634eb2509baf None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:37.444331 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f37bb4-c298-41c6-a061-634eb2509baf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:37.445472 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f37bb4-c298-41c6-a061-634eb2509baf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:37.446911 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f37bb4-c298-41c6-a061-634eb2509baf None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:37.448491 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 107/214] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:02:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:37.453975 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f37bb4-c298-41c6-a061-634eb2509baf None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:37.489743 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:37.507944 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf05c7b-5681-4e7c-91ef-2287c1d63f7f None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:37.545437 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7ba94e-c285-4ab5-8ea6-2af117c5e5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:37.546301 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ba94e-c285-4ab5-8ea6-2af117c5e5a3 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:37.546982 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ba94e-c285-4ab5-8ea6-2af117c5e5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:37.547533 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ba94e-c285-4ab5-8ea6-2af117c5e5a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:37.548875 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7ba94e-c285-4ab5-8ea6-2af117c5e5a3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:37.549187 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7ba94e-c285-4ab5-8ea6-2af117c5e5a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:37.550715 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7ba94e-c285-4ab5-8ea6-2af117c5e5a3 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:37.555801 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f37bb4-c298-41c6-a061-634eb2509baf None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:37.562469 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 108/215] 10.222.0.43 () {56 vars in 1101 bytes} [Tue Sep 6 16:02:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:02:37.567972 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7ba94e-c285-4ab5-8ea6-2af117c5e5a3 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:37.596130 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8b02f1-4c85-4bed-84d6-5b46fe58f816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:37.596779 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b02f1-4c85-4bed-84d6-5b46fe58f816 None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:37.597062 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b02f1-4c85-4bed-84d6-5b46fe58f816 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:37.597336 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b02f1-4c85-4bed-84d6-5b46fe58f816 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:37.597999 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8b02f1-4c85-4bed-84d6-5b46fe58f816 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:37.599468 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8b02f1-4c85-4bed-84d6-5b46fe58f816 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:37.614907 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:37.624583 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 108/216] 10.222.0.43 () {60 vars in 1132 bytes} [Tue Sep 6 16:02:37 2022] POST /identity/v3/users => generated 297 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:02:37.688260 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7ba94e-c285-4ab5-8ea6-2af117c5e5a3 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:37.726347 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a38cbd2-f6f1-46ae-a97a-89829cf5d7e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:37.727009 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a38cbd2-f6f1-46ae-a97a-89829cf5d7e9 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:37.727289 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a38cbd2-f6f1-46ae-a97a-89829cf5d7e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:37.727561 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a38cbd2-f6f1-46ae-a97a-89829cf5d7e9 None admin] PATH_INFO: `/v3/users/74003d65e29b414f8fe73e69d5b3ce4b` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:37.743550 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a38cbd2-f6f1-46ae-a97a-89829cf5d7e9 None admin] RBAC: Authorizing `identity:get_user(user_id=74003d65e29b414f8fe73e69d5b3ce4b)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:37.745004 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a38cbd2-f6f1-46ae-a97a-89829cf5d7e9 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:37.747075 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 109/217] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 6 16:02:37 2022] GET /identity/v3/users/74003d65e29b414f8fe73e69d5b3ce4b => generated 297 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:02:37.753150 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8b02f1-4c85-4bed-84d6-5b46fe58f816 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:37.871305 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:37.884267 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8b02f1-4c85-4bed-84d6-5b46fe58f816 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:37.923709 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce23b9a7-c955-4a99-b8ae-91f38852d412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:37.925159 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce23b9a7-c955-4a99-b8ae-91f38852d412 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:37.925438 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce23b9a7-c955-4a99-b8ae-91f38852d412 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:37.925707 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce23b9a7-c955-4a99-b8ae-91f38852d412 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:37.995167 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce23b9a7-c955-4a99-b8ae-91f38852d412 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:37.996595 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce23b9a7-c955-4a99-b8ae-91f38852d412 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:38.092610 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:38.103576 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 109/218] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 6 16:02:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 16:02:38.112781 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a38cbd2-f6f1-46ae-a97a-89829cf5d7e9 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:38.237707 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:38.245896 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a38cbd2-f6f1-46ae-a97a-89829cf5d7e9 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:38.292136 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b860882f-ef7a-42bf-a603-fa20efc11b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:38.292808 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b860882f-ef7a-42bf-a603-fa20efc11b23 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:38.293086 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b860882f-ef7a-42bf-a603-fa20efc11b23 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:38.293364 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b860882f-ef7a-42bf-a603-fa20efc11b23 None admin] PATH_INFO: `/v3/projects/cf9744296b5341f08017259b44d86f94` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:38.294491 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b860882f-ef7a-42bf-a603-fa20efc11b23 None admin] RBAC: Authorizing `identity:get_project(project_id=cf9744296b5341f08017259b44d86f94)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:38.295966 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b860882f-ef7a-42bf-a603-fa20efc11b23 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:38.298086 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 110/219] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 6 16:02:38 2022] GET /identity/v3/projects/cf9744296b5341f08017259b44d86f94 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:38.303999 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce23b9a7-c955-4a99-b8ae-91f38852d412 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:38.408216 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce23b9a7-c955-4a99-b8ae-91f38852d412 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:38.443864 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebddacc-0e1c-4361-bb05-a7871b2df29b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:38.444515 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebddacc-0e1c-4361-bb05-a7871b2df29b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:38.444812 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebddacc-0e1c-4361-bb05-a7871b2df29b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:38.445086 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebddacc-0e1c-4361-bb05-a7871b2df29b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:38.445811 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebddacc-0e1c-4361-bb05-a7871b2df29b None admin] RBAC: Adding query filter params (role.id=15252b0b89894c1e8a6bd9a98b1c3ab7, scope.project.id=cf9744296b5341f08017259b44d86f94, user.id=74003d65e29b414f8fe73e69d5b3ce4b) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:38.446117 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebddacc-0e1c-4361-bb05-a7871b2df29b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:38.447560 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebddacc-0e1c-4361-bb05-a7871b2df29b None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:38.453488 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 110/220] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 6 16:02:38 2022] GET /identity/v3/role_assignments?user.id=74003d65e29b414f8fe73e69d5b3ce4b&scope.project.id=cf9744296b5341f08017259b44d86f94&role.id=15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:02:38.583623 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281b7038-6cc3-4be2-9c0e-6f362d838c8d None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:38.584295 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281b7038-6cc3-4be2-9c0e-6f362d838c8d None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:38.584295 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281b7038-6cc3-4be2-9c0e-6f362d838c8d None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:38.585904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 111/221] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:02:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:02:38.591092 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb612627-65fa-426a-98ec-01568dbc71aa None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:38.591658 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb612627-65fa-426a-98ec-01568dbc71aa None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:38.592120 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb612627-65fa-426a-98ec-01568dbc71aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:38.613110 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-fb612627-65fa-426a-98ec-01568dbc71aa None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:38.614651 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-fb612627-65fa-426a-98ec-01568dbc71aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:38.671363 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:38.688494 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 111/222] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:02:38.758963 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f47b9-ded7-42c9-bffd-285cfb14396e None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:38.759608 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f47b9-ded7-42c9-bffd-285cfb14396e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:38.760074 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f47b9-ded7-42c9-bffd-285cfb14396e None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:38.761942 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 112/223] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:02:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:02:38.767367 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e31d78-1f7f-4543-b4cf-ea48bd5f7c79 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:38.767651 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e31d78-1f7f-4543-b4cf-ea48bd5f7c79 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:38.767920 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e31d78-1f7f-4543-b4cf-ea48bd5f7c79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:38.789796 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-25e31d78-1f7f-4543-b4cf-ea48bd5f7c79 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:38.790995 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-25e31d78-1f7f-4543-b4cf-ea48bd5f7c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:38.881903 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 112/224] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:02:39.069698 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5f47b9-ded7-42c9-bffd-285cfb14396e None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:39.195227 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5f47b9-ded7-42c9-bffd-285cfb14396e None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:39.242510 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff252c7f-c811-46c2-8e56-bb72dbdd35ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:39.243981 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff252c7f-c811-46c2-8e56-bb72dbdd35ff None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:39.244471 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff252c7f-c811-46c2-8e56-bb72dbdd35ff None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:39.244958 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff252c7f-c811-46c2-8e56-bb72dbdd35ff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:39.264444 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff252c7f-c811-46c2-8e56-bb72dbdd35ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:02:39.269672 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-ff252c7f-c811-46c2-8e56-bb72dbdd35ff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:02:39.269672 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:39.269672 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:39.269672 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:39.269672 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:39.269672 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:39.269672 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:39.269672 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:39.269672 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:39.269672 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:39.269672 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:39.269672 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:39.269672 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:02:39.269672 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:02:39.269672 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:02:39.269672 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:02:39.270524 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:02:39.270524 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:02:39.270524 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:02:39.270524 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:02:39.270524 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:39.270524 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:39.270524 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:39.270524 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:39.270524 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:39.270524 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:39.270524 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:39.270524 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:39.270524 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:39.270524 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:39.270524 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:39.270524 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:39.271312 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:39.271312 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:39.271312 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:39.271312 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:39.271312 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:02:39.271312 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:02:39.271312 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:02:39.271312 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:02:39.271312 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:02:39.271312 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:02:39.271312 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:02:39.271312 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:02:39.278721 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 113/225] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 6 16:02:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:02:39.280226 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e31d78-1f7f-4543-b4cf-ea48bd5f7c79 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:39.297757 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff252c7f-c811-46c2-8e56-bb72dbdd35ff None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:39.346806 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:39.410333 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e31d78-1f7f-4543-b4cf-ea48bd5f7c79 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:39.431477 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:39.454921 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542d0788-a9d4-4bf5-808f-8d4298bec736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:39.455758 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d0788-a9d4-4bf5-808f-8d4298bec736 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:39.456169 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d0788-a9d4-4bf5-808f-8d4298bec736 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:39.456983 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d0788-a9d4-4bf5-808f-8d4298bec736 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:39.460894 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542d0788-a9d4-4bf5-808f-8d4298bec736 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:39.461353 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542d0788-a9d4-4bf5-808f-8d4298bec736 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:39.462873 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542d0788-a9d4-4bf5-808f-8d4298bec736 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:39.477805 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 113/226] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 6 16:02:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:39.485098 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542d0788-a9d4-4bf5-808f-8d4298bec736 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:39.525165 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff252c7f-c811-46c2-8e56-bb72dbdd35ff None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:39.589504 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d5bf43-4dbd-430e-996e-2183776ba1b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:39.590137 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d5bf43-4dbd-430e-996e-2183776ba1b8 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:39.590409 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d5bf43-4dbd-430e-996e-2183776ba1b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:39.590681 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d5bf43-4dbd-430e-996e-2183776ba1b8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:39.617623 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d5bf43-4dbd-430e-996e-2183776ba1b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:02:39.622925 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-87d5bf43-4dbd-430e-996e-2183776ba1b8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 16:02:39.622925 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:39.622925 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:39.622925 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:39.622925 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:39.622925 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:39.622925 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:39.622925 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:39.622925 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:39.622925 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:39.622925 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:39.622925 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:39.622925 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:02:39.622925 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:02:39.622925 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:02:39.622925 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:02:39.623781 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:02:39.623781 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:02:39.623781 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:02:39.623781 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:02:39.623781 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:39.623781 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:39.623781 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:39.623781 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:39.623781 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:39.623781 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:39.623781 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:39.623781 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:39.623781 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:39.623781 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:39.623781 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:39.623781 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:39.624569 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:39.624569 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:39.624569 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:39.624569 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:39.624569 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:02:39.624569 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:02:39.624569 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:02:39.624569 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:02:39.624569 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:02:39.624569 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:02:39.624569 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 16:02:39.624569 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:02:39.627146 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 114/227] 10.222.0.43 () {56 vars in 1114 bytes} [Tue Sep 6 16:02:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:02:39.633098 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d5bf43-4dbd-430e-996e-2183776ba1b8 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:39.642014 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542d0788-a9d4-4bf5-808f-8d4298bec736 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:39.678954 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52b66ad-0c91-4fe5-853c-591cb6dd31f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:39.679966 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52b66ad-0c91-4fe5-853c-591cb6dd31f0 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:39.680666 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52b66ad-0c91-4fe5-853c-591cb6dd31f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:39.681287 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52b66ad-0c91-4fe5-853c-591cb6dd31f0 None admin] PATH_INFO: `/v3/users/a73439da3d3a43ca95973b14c17f910f` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:39.682780 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52b66ad-0c91-4fe5-853c-591cb6dd31f0 None admin] RBAC: Authorizing `identity:get_user(user_id=a73439da3d3a43ca95973b14c17f910f)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:39.684716 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52b66ad-0c91-4fe5-853c-591cb6dd31f0 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:39.686568 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 114/228] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 6 16:02:39 2022] GET /identity/v3/users/a73439da3d3a43ca95973b14c17f910f => generated 283 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:02:39.692262 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52b66ad-0c91-4fe5-853c-591cb6dd31f0 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:39.746952 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:39.811687 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52b66ad-0c91-4fe5-853c-591cb6dd31f0 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:39.826522 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:39.851615 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c16f797-2d77-4da4-8f1b-4d559a097af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:39.852638 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c16f797-2d77-4da4-8f1b-4d559a097af0 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:39.853146 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c16f797-2d77-4da4-8f1b-4d559a097af0 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:39.853770 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c16f797-2d77-4da4-8f1b-4d559a097af0 None admin] PATH_INFO: `/v3/projects/cf9744296b5341f08017259b44d86f94` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:39.855147 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c16f797-2d77-4da4-8f1b-4d559a097af0 None admin] RBAC: Authorizing `identity:get_project(project_id=cf9744296b5341f08017259b44d86f94)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:39.856909 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c16f797-2d77-4da4-8f1b-4d559a097af0 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:39.859242 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 115/229] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 6 16:02:39 2022] GET /identity/v3/projects/cf9744296b5341f08017259b44d86f94 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:02:39.873986 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c16f797-2d77-4da4-8f1b-4d559a097af0 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:39.882548 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d5bf43-4dbd-430e-996e-2183776ba1b8 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:39.928723 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e2dbb1-a2d5-42c3-8b3e-b6781873aad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:39.930729 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e2dbb1-a2d5-42c3-8b3e-b6781873aad4 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:39.933278 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e2dbb1-a2d5-42c3-8b3e-b6781873aad4 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:39.933764 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e2dbb1-a2d5-42c3-8b3e-b6781873aad4 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:39.934688 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e2dbb1-a2d5-42c3-8b3e-b6781873aad4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:39.936531 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e2dbb1-a2d5-42c3-8b3e-b6781873aad4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:39.938242 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e2dbb1-a2d5-42c3-8b3e-b6781873aad4 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:39.960520 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 115/230] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 6 16:02:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:02:39.966225 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e2dbb1-a2d5-42c3-8b3e-b6781873aad4 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:40.036644 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c16f797-2d77-4da4-8f1b-4d559a097af0 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:40.053380 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:40.084430 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc083b6-5efb-435f-973d-89b574f93a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:40.086699 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc083b6-5efb-435f-973d-89b574f93a94 None admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:40.086993 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc083b6-5efb-435f-973d-89b574f93a94 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:40.087265 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc083b6-5efb-435f-973d-89b574f93a94 None admin] PATH_INFO: `/v3/projects/cf9744296b5341f08017259b44d86f94/users/a73439da3d3a43ca95973b14c17f910f/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:40.091676 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc083b6-5efb-435f-973d-89b574f93a94 None admin] RBAC: Authorizing `identity:create_grant(project_id=cf9744296b5341f08017259b44d86f94, user_id=a73439da3d3a43ca95973b14c17f910f, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:40.095884 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc083b6-5efb-435f-973d-89b574f93a94 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:40.129141 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 116/231] 10.222.0.43 () {58 vars in 1439 bytes} [Tue Sep 6 16:02:39 2022] PUT /identity/v3/projects/cf9744296b5341f08017259b44d86f94/users/a73439da3d3a43ca95973b14c17f910f/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:02:40.186674 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e2dbb1-a2d5-42c3-8b3e-b6781873aad4 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:40.197800 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52948591-0f4d-407a-a5d4-a3fd35731826 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:40.198133 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52948591-0f4d-407a-a5d4-a3fd35731826 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:40.198367 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52948591-0f4d-407a-a5d4-a3fd35731826 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:40.200735 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 117/232] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:02:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:02:40.205695 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61223b29-c1e1-4361-b9f2-c5394771e5a9 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:40.205975 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61223b29-c1e1-4361-b9f2-c5394771e5a9 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:40.206247 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61223b29-c1e1-4361-b9f2-c5394771e5a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:40.245015 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-61223b29-c1e1-4361-b9f2-c5394771e5a9 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:40.245015 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-61223b29-c1e1-4361-b9f2-c5394771e5a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:40.252809 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5109ff1-b047-4276-9c13-40d2621bdfb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:40.252809 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5109ff1-b047-4276-9c13-40d2621bdfb6 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:40.252809 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5109ff1-b047-4276-9c13-40d2621bdfb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:40.252809 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5109ff1-b047-4276-9c13-40d2621bdfb6 None admin] PATH_INFO: `/v3/users/a73439da3d3a43ca95973b14c17f910f` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:40.253212 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5109ff1-b047-4276-9c13-40d2621bdfb6 None admin] RBAC: Authorizing `identity:get_user(user_id=a73439da3d3a43ca95973b14c17f910f)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:40.253212 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5109ff1-b047-4276-9c13-40d2621bdfb6 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:40.254331 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 116/233] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 6 16:02:39 2022] GET /identity/v3/users/a73439da3d3a43ca95973b14c17f910f => generated 283 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:02:40.260645 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5109ff1-b047-4276-9c13-40d2621bdfb6 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:40.289289 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:40.382182 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 118/234] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 16:02:40.392792 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5109ff1-b047-4276-9c13-40d2621bdfb6 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:40.433851 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187a6d7c-509a-418e-b00d-f6983c01fb6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:40.434501 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187a6d7c-509a-418e-b00d-f6983c01fb6a None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:40.434779 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187a6d7c-509a-418e-b00d-f6983c01fb6a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:40.435060 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187a6d7c-509a-418e-b00d-f6983c01fb6a None admin] PATH_INFO: `/v3/projects/cf9744296b5341f08017259b44d86f94` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:40.436189 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187a6d7c-509a-418e-b00d-f6983c01fb6a None admin] RBAC: Authorizing `identity:get_project(project_id=cf9744296b5341f08017259b44d86f94)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:40.437764 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187a6d7c-509a-418e-b00d-f6983c01fb6a None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:40.439784 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 117/235] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 6 16:02:40 2022] GET /identity/v3/projects/cf9744296b5341f08017259b44d86f94 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:02:40.445340 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61223b29-c1e1-4361-b9f2-c5394771e5a9 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:40.485913 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7296ae-6619-4400-8fc9-4fb3ba48387c None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:40.486529 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7296ae-6619-4400-8fc9-4fb3ba48387c None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:40.486998 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7296ae-6619-4400-8fc9-4fb3ba48387c None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:40.489021 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 118/236] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:02:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 16:02:40.493935 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7460dbf-3630-4c97-8505-1ab0c58b266b None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:40.494364 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7460dbf-3630-4c97-8505-1ab0c58b266b None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:40.495938 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7460dbf-3630-4c97-8505-1ab0c58b266b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:40.515500 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-f7460dbf-3630-4c97-8505-1ab0c58b266b None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:40.517419 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f7460dbf-3630-4c97-8505-1ab0c58b266b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:40.610033 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61223b29-c1e1-4361-b9f2-c5394771e5a9 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:40.615663 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:40.654170 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 119/237] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 16:02:40.672259 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873d0cfa-d3a4-4ba3-924a-37c4546c15d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:40.673120 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d0cfa-d3a4-4ba3-924a-37c4546c15d6 None admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:40.673463 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d0cfa-d3a4-4ba3-924a-37c4546c15d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:40.673742 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d0cfa-d3a4-4ba3-924a-37c4546c15d6 None admin] PATH_INFO: `/v3/projects/cf9744296b5341f08017259b44d86f94/users/a73439da3d3a43ca95973b14c17f910f/roles/61df19b017494bdcadc9a5e9d8bb9e52` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:40.677225 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873d0cfa-d3a4-4ba3-924a-37c4546c15d6 None admin] RBAC: Authorizing `identity:create_grant(project_id=cf9744296b5341f08017259b44d86f94, user_id=a73439da3d3a43ca95973b14c17f910f, role_id=61df19b017494bdcadc9a5e9d8bb9e52)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:40.679145 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873d0cfa-d3a4-4ba3-924a-37c4546c15d6 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:40.703124 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 119/238] 10.222.0.43 () {58 vars in 1439 bytes} [Tue Sep 6 16:02:40 2022] PUT /identity/v3/projects/cf9744296b5341f08017259b44d86f94/users/a73439da3d3a43ca95973b14c17f910f/roles/61df19b017494bdcadc9a5e9d8bb9e52 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:02:40.729015 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7460dbf-3630-4c97-8505-1ab0c58b266b None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:40.747960 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:40.877616 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7460dbf-3630-4c97-8505-1ab0c58b266b None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:40.922129 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7700736-c1b5-470c-8359-87842dfd6a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:40.922784 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7700736-c1b5-470c-8359-87842dfd6a4d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:40.923118 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7700736-c1b5-470c-8359-87842dfd6a4d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:40.923394 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7700736-c1b5-470c-8359-87842dfd6a4d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:40.931400 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da55afba-671c-4214-afd2-1f54a82ebc75 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:40.931682 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da55afba-671c-4214-afd2-1f54a82ebc75 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:40.931961 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da55afba-671c-4214-afd2-1f54a82ebc75 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:40.933606 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 120/239] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:02:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:02:40.940124 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7700736-c1b5-470c-8359-87842dfd6a4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:02:40.941924 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0c929f-13fd-4ca9-a1b9-1393f9779899 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:40.942315 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0c929f-13fd-4ca9-a1b9-1393f9779899 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:40.942688 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0c929f-13fd-4ca9-a1b9-1393f9779899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:40.945904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-d7700736-c1b5-470c-8359-87842dfd6a4d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 16:02:40.945904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:40.945904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:40.945904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:40.945904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:40.945904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:40.945904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:40.945904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:40.945904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:40.945904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:40.945904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:40.945904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:40.945904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:02:40.945904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:02:40.945904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:02:40.945904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:02:40.946943 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:02:40.946943 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:02:40.946943 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:02:40.946943 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:02:40.946943 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:40.946943 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:40.946943 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:40.946943 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:40.946943 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:40.946943 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:40.946943 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:40.946943 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:40.946943 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:40.946943 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:40.946943 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:40.946943 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:40.947911 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:40.947911 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:40.947911 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:40.947911 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:40.947911 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:02:40.947911 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:02:40.947911 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:02:40.947911 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:02:40.947911 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:02:40.947911 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:02:40.947911 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 16:02:40.947911 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:02:40.949987 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 120/240] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 6 16:02:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:02:40.955689 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7700736-c1b5-470c-8359-87842dfd6a4d None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:40.967784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-9e0c929f-13fd-4ca9-a1b9-1393f9779899 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:40.969156 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-9e0c929f-13fd-4ca9-a1b9-1393f9779899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:41.039848 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:41.071041 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7700736-c1b5-470c-8359-87842dfd6a4d None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:41.073022 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 121/241] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 16:02:41.075027 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0c929f-13fd-4ca9-a1b9-1393f9779899 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:41.107118 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716e8a37-c281-4fa9-9d85-be3de4764709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:41.107741 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716e8a37-c281-4fa9-9d85-be3de4764709 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:41.108018 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716e8a37-c281-4fa9-9d85-be3de4764709 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:41.108288 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716e8a37-c281-4fa9-9d85-be3de4764709 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:41.109062 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716e8a37-c281-4fa9-9d85-be3de4764709 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:41.109364 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716e8a37-c281-4fa9-9d85-be3de4764709 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:41.110831 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716e8a37-c281-4fa9-9d85-be3de4764709 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:41.119387 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 121/242] 10.222.0.43 () {56 vars in 1101 bytes} [Tue Sep 6 16:02:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:02:41.125777 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716e8a37-c281-4fa9-9d85-be3de4764709 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:41.184273 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:41.192468 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0c929f-13fd-4ca9-a1b9-1393f9779899 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:41.231858 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4365dafe-9827-47c1-9cc9-672cffccf40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:41.232833 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4365dafe-9827-47c1-9cc9-672cffccf40a None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:41.233303 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4365dafe-9827-47c1-9cc9-672cffccf40a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:41.233741 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4365dafe-9827-47c1-9cc9-672cffccf40a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:41.235040 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4365dafe-9827-47c1-9cc9-672cffccf40a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:41.237097 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4365dafe-9827-47c1-9cc9-672cffccf40a None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:41.239467 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 122/243] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:02:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:41.251750 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4365dafe-9827-47c1-9cc9-672cffccf40a None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:41.365556 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4365dafe-9827-47c1-9cc9-672cffccf40a None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:41.385632 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716e8a37-c281-4fa9-9d85-be3de4764709 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:41.403141 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec17b9-e13a-4f54-8ee8-e23338fe55fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:41.404360 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec17b9-e13a-4f54-8ee8-e23338fe55fb None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:41.405186 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec17b9-e13a-4f54-8ee8-e23338fe55fb None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:41.405659 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec17b9-e13a-4f54-8ee8-e23338fe55fb None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:41.406888 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fec17b9-e13a-4f54-8ee8-e23338fe55fb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:41.409313 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fec17b9-e13a-4f54-8ee8-e23338fe55fb None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:41.443670 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 123/244] 10.222.0.43 () {60 vars in 1132 bytes} [Tue Sep 6 16:02:41 2022] POST /identity/v3/users => generated 293 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:02:41.446341 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb1caf7-f5f5-4ed1-aed8-8835c1f13186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:41.447266 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb1caf7-f5f5-4ed1-aed8-8835c1f13186 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:41.447770 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb1caf7-f5f5-4ed1-aed8-8835c1f13186 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:41.448265 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb1caf7-f5f5-4ed1-aed8-8835c1f13186 None admin] PATH_INFO: `/v3/users/c44807cbc8c24276b7a6496957d3330c` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:41.457476 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec17b9-e13a-4f54-8ee8-e23338fe55fb None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:41.469179 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:41.484621 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb1caf7-f5f5-4ed1-aed8-8835c1f13186 None admin] RBAC: Authorizing `identity:get_user(user_id=c44807cbc8c24276b7a6496957d3330c)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:41.487465 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb1caf7-f5f5-4ed1-aed8-8835c1f13186 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:41.490357 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 122/245] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 6 16:02:41 2022] GET /identity/v3/users/c44807cbc8c24276b7a6496957d3330c => generated 297 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:41.497879 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:41.498077 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb1caf7-f5f5-4ed1-aed8-8835c1f13186 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:41.648149 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb1caf7-f5f5-4ed1-aed8-8835c1f13186 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:41.650432 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec17b9-e13a-4f54-8ee8-e23338fe55fb None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:41.684061 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6d3413-4a81-45de-af38-8dd3c18246c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:41.684673 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6d3413-4a81-45de-af38-8dd3c18246c7 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:41.684977 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6d3413-4a81-45de-af38-8dd3c18246c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:41.685251 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6d3413-4a81-45de-af38-8dd3c18246c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:41.690596 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78921cb1-9241-436f-b949-63592028f6ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:41.691248 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78921cb1-9241-436f-b949-63592028f6ae None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:41.691525 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78921cb1-9241-436f-b949-63592028f6ae None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:41.691798 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78921cb1-9241-436f-b949-63592028f6ae None admin] PATH_INFO: `/v3/roles/member` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:41.703716 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78921cb1-9241-436f-b949-63592028f6ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:02:41.710586 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-78921cb1-9241-436f-b949-63592028f6ae None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 16:02:41.710586 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:41.710586 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:41.710586 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:41.710586 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:41.710586 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:41.710586 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:41.710586 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:41.710586 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:41.710586 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:41.710586 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:41.710586 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:41.710586 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:02:41.710586 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:02:41.710586 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:02:41.710586 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:02:41.711511 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:02:41.711511 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:02:41.711511 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:02:41.711511 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:02:41.711511 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:41.711511 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:41.711511 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:41.711511 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:41.711511 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:41.711511 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:41.711511 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:41.711511 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:41.711511 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:41.711511 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:41.711511 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:41.711511 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:41.712290 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:41.712290 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:41.712290 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:41.712290 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:41.712290 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:02:41.712290 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:02:41.712290 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:02:41.712290 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:02:41.712290 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:02:41.712290 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:02:41.712290 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 16:02:41.712290 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:02:41.712290 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 124/246] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 6 16:02:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:02:41.717561 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78921cb1-9241-436f-b949-63592028f6ae None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:41.747706 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:41.757710 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6d3413-4a81-45de-af38-8dd3c18246c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:41.759227 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6d3413-4a81-45de-af38-8dd3c18246c7 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:41.813231 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:41.840554 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78921cb1-9241-436f-b949-63592028f6ae None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:41.863493 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 123/247] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 6 16:02:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 16:02:41.869985 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6d3413-4a81-45de-af38-8dd3c18246c7 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:41.878906 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2458e663-da57-4658-8a46-0ffdfae57f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:41.879754 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2458e663-da57-4658-8a46-0ffdfae57f3c None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:41.880167 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2458e663-da57-4658-8a46-0ffdfae57f3c None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:41.880570 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2458e663-da57-4658-8a46-0ffdfae57f3c None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:41.881400 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2458e663-da57-4658-8a46-0ffdfae57f3c None admin] RBAC: Adding query filter params (name=member) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:41.881973 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2458e663-da57-4658-8a46-0ffdfae57f3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:41.883518 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2458e663-da57-4658-8a46-0ffdfae57f3c None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:41.892802 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 125/248] 10.222.0.43 () {56 vars in 1101 bytes} [Tue Sep 6 16:02:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:02:41.898522 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2458e663-da57-4658-8a46-0ffdfae57f3c None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:41.979005 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:41.988483 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6d3413-4a81-45de-af38-8dd3c18246c7 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:42.034327 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2458e663-da57-4658-8a46-0ffdfae57f3c None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:42.041971 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd1ed66-a8ec-433f-a1cc-dac00f99d442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:42.042631 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd1ed66-a8ec-433f-a1cc-dac00f99d442 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:42.042903 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd1ed66-a8ec-433f-a1cc-dac00f99d442 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:42.043173 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd1ed66-a8ec-433f-a1cc-dac00f99d442 None admin] PATH_INFO: `/v3/projects/cf9744296b5341f08017259b44d86f94` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:42.044331 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd1ed66-a8ec-433f-a1cc-dac00f99d442 None admin] RBAC: Authorizing `identity:get_project(project_id=cf9744296b5341f08017259b44d86f94)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:42.045851 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd1ed66-a8ec-433f-a1cc-dac00f99d442 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:42.048572 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 124/249] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 6 16:02:41 2022] GET /identity/v3/projects/cf9744296b5341f08017259b44d86f94 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:02:42.054004 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd1ed66-a8ec-433f-a1cc-dac00f99d442 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:42.084873 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f3a225-fd6a-4c93-87d5-25067bd2ea85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:42.085517 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f3a225-fd6a-4c93-87d5-25067bd2ea85 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:42.085793 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f3a225-fd6a-4c93-87d5-25067bd2ea85 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:42.086062 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f3a225-fd6a-4c93-87d5-25067bd2ea85 None admin] PATH_INFO: `/v3/users/74003d65e29b414f8fe73e69d5b3ce4b` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:42.087322 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f3a225-fd6a-4c93-87d5-25067bd2ea85 None admin] RBAC: Authorizing `identity:get_user(user_id=74003d65e29b414f8fe73e69d5b3ce4b)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:42.088925 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f3a225-fd6a-4c93-87d5-25067bd2ea85 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:42.090783 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 126/250] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 6 16:02:41 2022] GET /identity/v3/users/74003d65e29b414f8fe73e69d5b3ce4b => generated 297 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:02:42.097423 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f3a225-fd6a-4c93-87d5-25067bd2ea85 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:42.113364 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:42.194951 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd1ed66-a8ec-433f-a1cc-dac00f99d442 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:42.219245 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f3a225-fd6a-4c93-87d5-25067bd2ea85 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:42.240272 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67120400-0a61-48e9-976b-04892ed2ee76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:42.242133 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67120400-0a61-48e9-976b-04892ed2ee76 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:42.243343 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67120400-0a61-48e9-976b-04892ed2ee76 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:42.243921 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67120400-0a61-48e9-976b-04892ed2ee76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:42.244914 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67120400-0a61-48e9-976b-04892ed2ee76 None admin] RBAC: Adding query filter params (role.id=4908d6a02c4049e88249e9afbf7e2e9f, scope.project.id=cf9744296b5341f08017259b44d86f94, user.id=c44807cbc8c24276b7a6496957d3330c) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:42.245394 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67120400-0a61-48e9-976b-04892ed2ee76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:42.248418 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67120400-0a61-48e9-976b-04892ed2ee76 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:42.262912 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 125/251] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 6 16:02:42 2022] GET /identity/v3/role_assignments?user.id=c44807cbc8c24276b7a6496957d3330c&scope.project.id=cf9744296b5341f08017259b44d86f94&role.id=4908d6a02c4049e88249e9afbf7e2e9f => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:02:42.271682 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2e9651-8855-4656-bc9f-d884793553c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:42.272705 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e9651-8855-4656-bc9f-d884793553c8 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:42.273160 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e9651-8855-4656-bc9f-d884793553c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:42.273571 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e9651-8855-4656-bc9f-d884793553c8 None admin] PATH_INFO: `/v3/projects/cf9744296b5341f08017259b44d86f94` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:42.274990 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2e9651-8855-4656-bc9f-d884793553c8 None admin] RBAC: Authorizing `identity:get_project(project_id=cf9744296b5341f08017259b44d86f94)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:42.276737 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2e9651-8855-4656-bc9f-d884793553c8 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:42.279592 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 127/252] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 6 16:02:42 2022] GET /identity/v3/projects/cf9744296b5341f08017259b44d86f94 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:02:42.285773 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67120400-0a61-48e9-976b-04892ed2ee76 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:42.316372 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:42.419067 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67120400-0a61-48e9-976b-04892ed2ee76 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:42.467517 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d87eb2e-7fac-4c95-8cc8-818699ebda15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:42.468479 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d87eb2e-7fac-4c95-8cc8-818699ebda15 None admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:42.469004 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d87eb2e-7fac-4c95-8cc8-818699ebda15 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:42.469418 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d87eb2e-7fac-4c95-8cc8-818699ebda15 None admin] PATH_INFO: `/v3/projects/cf9744296b5341f08017259b44d86f94/users/74003d65e29b414f8fe73e69d5b3ce4b/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:42.471494 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d87eb2e-7fac-4c95-8cc8-818699ebda15 None admin] RBAC: Authorizing `identity:create_grant(project_id=cf9744296b5341f08017259b44d86f94, user_id=74003d65e29b414f8fe73e69d5b3ce4b, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:42.473151 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d87eb2e-7fac-4c95-8cc8-818699ebda15 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:42.483603 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46a1a0f-6e8b-4d01-b68f-c67f8d0f9b43 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:42.484144 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46a1a0f-6e8b-4d01-b68f-c67f8d0f9b43 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:42.484144 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46a1a0f-6e8b-4d01-b68f-c67f8d0f9b43 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:42.485838 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 128/253] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:02:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:02:42.497774 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9472da6-1b62-4265-afea-b0a36e57d110 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:42.497965 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9472da6-1b62-4265-afea-b0a36e57d110 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:42.498287 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9472da6-1b62-4265-afea-b0a36e57d110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:42.506275 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 126/254] 10.222.0.43 () {58 vars in 1439 bytes} [Tue Sep 6 16:02:42 2022] PUT /identity/v3/projects/cf9744296b5341f08017259b44d86f94/users/74003d65e29b414f8fe73e69d5b3ce4b/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:02:42.523851 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-c9472da6-1b62-4265-afea-b0a36e57d110 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:42.525118 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c9472da6-1b62-4265-afea-b0a36e57d110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:42.623194 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 129/255] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:02:42.958772 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d87eb2e-7fac-4c95-8cc8-818699ebda15 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:43.070490 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:43.090149 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d87eb2e-7fac-4c95-8cc8-818699ebda15 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:43.132649 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725576e0-1593-497e-ad67-f4d858d4ebd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:43.133332 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725576e0-1593-497e-ad67-f4d858d4ebd7 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:43.133608 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725576e0-1593-497e-ad67-f4d858d4ebd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:43.133970 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725576e0-1593-497e-ad67-f4d858d4ebd7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:43.149411 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725576e0-1593-497e-ad67-f4d858d4ebd7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:02:43.154865 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-725576e0-1593-497e-ad67-f4d858d4ebd7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:02:43.154865 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:43.154865 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:43.154865 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:43.154865 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:43.154865 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:43.154865 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:43.154865 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:43.154865 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:43.154865 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:43.154865 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:43.154865 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:43.154865 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:02:43.154865 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:02:43.154865 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:02:43.154865 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:02:43.155757 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:02:43.155757 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:02:43.155757 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:02:43.155757 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:02:43.155757 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:43.155757 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:43.155757 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:43.155757 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:43.155757 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:43.155757 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:43.155757 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:43.155757 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:43.155757 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:43.155757 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:43.155757 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:43.155757 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:43.156559 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:43.156559 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:43.156559 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:43.156559 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:43.156559 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:02:43.156559 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:02:43.156559 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:02:43.156559 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:02:43.156559 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:02:43.156559 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:02:43.156559 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:02:43.156559 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:02:43.156559 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 127/256] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 6 16:02:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:02:43.161959 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9472da6-1b62-4265-afea-b0a36e57d110 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:43.171173 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:43.310020 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9472da6-1b62-4265-afea-b0a36e57d110 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:43.362424 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0218121-3938-467e-a6d2-a40463b7364b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:43.363047 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0218121-3938-467e-a6d2-a40463b7364b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:43.363322 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0218121-3938-467e-a6d2-a40463b7364b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:43.363590 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0218121-3938-467e-a6d2-a40463b7364b None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:43.364264 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0218121-3938-467e-a6d2-a40463b7364b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:43.364633 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0218121-3938-467e-a6d2-a40463b7364b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:43.366077 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0218121-3938-467e-a6d2-a40463b7364b None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:43.379979 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 130/257] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 6 16:02:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:43.385926 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725576e0-1593-497e-ad67-f4d858d4ebd7 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:43.495411 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725576e0-1593-497e-ad67-f4d858d4ebd7 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:43.537201 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059b37f4-b486-47af-bd49-d950d696c3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:43.537881 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059b37f4-b486-47af-bd49-d950d696c3df None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:43.538150 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059b37f4-b486-47af-bd49-d950d696c3df None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:43.538411 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059b37f4-b486-47af-bd49-d950d696c3df None admin] PATH_INFO: `/v3/users/a73439da3d3a43ca95973b14c17f910f` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:43.539640 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059b37f4-b486-47af-bd49-d950d696c3df None admin] RBAC: Authorizing `identity:get_user(user_id=a73439da3d3a43ca95973b14c17f910f)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:43.541713 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059b37f4-b486-47af-bd49-d950d696c3df None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:43.543809 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 128/258] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 6 16:02:43 2022] GET /identity/v3/users/a73439da3d3a43ca95973b14c17f910f => generated 283 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:43.549593 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0218121-3938-467e-a6d2-a40463b7364b None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:43.580621 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a61ae68-2787-4aae-8ae7-8b2898fcda89 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:43.580971 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a61ae68-2787-4aae-8ae7-8b2898fcda89 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:43.581199 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a61ae68-2787-4aae-8ae7-8b2898fcda89 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:43.583032 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 129/259] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:02:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 16:02:43.588299 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf054432-3483-457d-95c8-5a7ae548473b None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:43.588958 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf054432-3483-457d-95c8-5a7ae548473b None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:43.588958 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf054432-3483-457d-95c8-5a7ae548473b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:43.610475 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-bf054432-3483-457d-95c8-5a7ae548473b None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:43.611658 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-bf054432-3483-457d-95c8-5a7ae548473b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:43.620241 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:43.636514 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:43.675963 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0218121-3938-467e-a6d2-a40463b7364b None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:43.707293 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 130/260] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 16:02:43.714748 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859c6d55-5cc6-46b2-89e6-63fd9b036354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:43.715435 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859c6d55-5cc6-46b2-89e6-63fd9b036354 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:43.715707 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859c6d55-5cc6-46b2-89e6-63fd9b036354 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:43.715981 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859c6d55-5cc6-46b2-89e6-63fd9b036354 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:43.786315 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859c6d55-5cc6-46b2-89e6-63fd9b036354 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:43.787918 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859c6d55-5cc6-46b2-89e6-63fd9b036354 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:43.855671 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:43.891545 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 131/261] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 6 16:02:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 16:02:43.896860 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf054432-3483-457d-95c8-5a7ae548473b None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:44.022746 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf054432-3483-457d-95c8-5a7ae548473b None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:44.061550 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77daa70-9f6e-40ee-a3d8-36c28b2fe2cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:44.062111 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77daa70-9f6e-40ee-a3d8-36c28b2fe2cb None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:44.062384 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77daa70-9f6e-40ee-a3d8-36c28b2fe2cb None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:44.062646 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77daa70-9f6e-40ee-a3d8-36c28b2fe2cb None admin] PATH_INFO: `/v3/projects/cf9744296b5341f08017259b44d86f94` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:44.063852 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77daa70-9f6e-40ee-a3d8-36c28b2fe2cb None admin] RBAC: Authorizing `identity:get_project(project_id=cf9744296b5341f08017259b44d86f94)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:44.065371 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77daa70-9f6e-40ee-a3d8-36c28b2fe2cb None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:44.067005 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 131/262] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 6 16:02:43 2022] GET /identity/v3/projects/cf9744296b5341f08017259b44d86f94 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:44.073151 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859c6d55-5cc6-46b2-89e6-63fd9b036354 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:44.090546 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77daa70-9f6e-40ee-a3d8-36c28b2fe2cb None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:44.103292 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:44.281666 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77daa70-9f6e-40ee-a3d8-36c28b2fe2cb None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:44.301797 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859c6d55-5cc6-46b2-89e6-63fd9b036354 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:44.358970 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e58d2fc-733a-40ae-a204-93ea7f8e2246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:44.359515 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e58d2fc-733a-40ae-a204-93ea7f8e2246 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:44.359799 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e58d2fc-733a-40ae-a204-93ea7f8e2246 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:44.360068 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e58d2fc-733a-40ae-a204-93ea7f8e2246 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:44.360828 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e58d2fc-733a-40ae-a204-93ea7f8e2246 None admin] RBAC: Adding query filter params (role.id=edaabdea8be84a7980fb2c651fe52f63, scope.project.id=cf9744296b5341f08017259b44d86f94, user.id=a73439da3d3a43ca95973b14c17f910f) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:44.361195 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e58d2fc-733a-40ae-a204-93ea7f8e2246 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:44.361437 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e96179-c083-4598-a50a-94102ddfd3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:44.362143 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e96179-c083-4598-a50a-94102ddfd3b6 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:44.362417 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e96179-c083-4598-a50a-94102ddfd3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:44.362641 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e58d2fc-733a-40ae-a204-93ea7f8e2246 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:44.362799 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e96179-c083-4598-a50a-94102ddfd3b6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:44.369488 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 132/263] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 6 16:02:44 2022] GET /identity/v3/role_assignments?user.id=a73439da3d3a43ca95973b14c17f910f&scope.project.id=cf9744296b5341f08017259b44d86f94&role.id=edaabdea8be84a7980fb2c651fe52f63 => generated 608 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:44.388075 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e96179-c083-4598-a50a-94102ddfd3b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:02:44.393036 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-34e96179-c083-4598-a50a-94102ddfd3b6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 16:02:44.393036 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:44.393036 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:44.393036 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:44.393036 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:44.393036 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:44.393036 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:44.393036 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:44.393036 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:44.393036 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:44.393036 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:44.393036 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:44.393036 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:02:44.393036 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:02:44.393036 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:02:44.393036 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:02:44.395057 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:02:44.395057 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:02:44.395057 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:02:44.395057 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:02:44.395057 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:44.395057 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:44.395057 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:44.395057 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:44.395057 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:44.395057 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:44.395057 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:44.395057 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:44.395057 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:44.395057 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:44.395057 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:44.395057 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:44.397092 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:44.397092 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:44.397092 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:44.397092 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:44.397092 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:02:44.397092 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:02:44.397092 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:02:44.397092 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:02:44.397092 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:02:44.397092 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:02:44.397092 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 16:02:44.397092 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:02:44.404792 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 132/264] 10.222.0.43 () {56 vars in 1114 bytes} [Tue Sep 6 16:02:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:02:44.410001 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e58d2fc-733a-40ae-a204-93ea7f8e2246 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:44.451445 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:44.550807 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e58d2fc-733a-40ae-a204-93ea7f8e2246 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:44.596398 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b19666f-2325-43c4-99bc-0365a6e3dee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:44.597075 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b19666f-2325-43c4-99bc-0365a6e3dee3 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:44.597355 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b19666f-2325-43c4-99bc-0365a6e3dee3 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:44.597631 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b19666f-2325-43c4-99bc-0365a6e3dee3 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:44.598408 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b19666f-2325-43c4-99bc-0365a6e3dee3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:44.598720 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b19666f-2325-43c4-99bc-0365a6e3dee3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:44.600267 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b19666f-2325-43c4-99bc-0365a6e3dee3 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:44.609890 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 133/265] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 6 16:02:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:44.616057 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e96179-c083-4598-a50a-94102ddfd3b6 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:44.733049 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:44.770335 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e96179-c083-4598-a50a-94102ddfd3b6 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:44.818745 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b54e60e-a64c-48d7-a583-cc8687bf3cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:44.819448 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b54e60e-a64c-48d7-a583-cc8687bf3cff None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:44.819724 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b54e60e-a64c-48d7-a583-cc8687bf3cff None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:44.819991 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b54e60e-a64c-48d7-a583-cc8687bf3cff None admin] PATH_INFO: `/v3/users/a73439da3d3a43ca95973b14c17f910f` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:44.825439 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b54e60e-a64c-48d7-a583-cc8687bf3cff None admin] RBAC: Authorizing `identity:get_user(user_id=a73439da3d3a43ca95973b14c17f910f)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:44.826952 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b54e60e-a64c-48d7-a583-cc8687bf3cff None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:44.828561 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 133/266] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 6 16:02:44 2022] GET /identity/v3/users/a73439da3d3a43ca95973b14c17f910f => generated 283 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:44.834878 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b19666f-2325-43c4-99bc-0365a6e3dee3 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:44.953898 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:44.963614 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b19666f-2325-43c4-99bc-0365a6e3dee3 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:45.011840 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcec785-a21a-48bf-9205-b696e501f8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:45.012475 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcec785-a21a-48bf-9205-b696e501f8d7 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:45.012773 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcec785-a21a-48bf-9205-b696e501f8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:45.013047 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcec785-a21a-48bf-9205-b696e501f8d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:45.043371 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd5d9e-bf3c-470a-8fa1-c45447337440 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:45.043885 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd5d9e-bf3c-470a-8fa1-c45447337440 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:45.043885 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd5d9e-bf3c-470a-8fa1-c45447337440 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:45.046572 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 134/267] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:02:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:02:45.051240 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24fb550-6572-4943-89fc-3ad06553a580 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:45.051521 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24fb550-6572-4943-89fc-3ad06553a580 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:45.051792 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24fb550-6572-4943-89fc-3ad06553a580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:45.091510 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcec785-a21a-48bf-9205-b696e501f8d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:45.093561 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcec785-a21a-48bf-9205-b696e501f8d7 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:45.098318 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-a24fb550-6572-4943-89fc-3ad06553a580 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:45.099515 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a24fb550-6572-4943-89fc-3ad06553a580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:45.182608 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 135/268] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:02:45.186666 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b359a0-2c9b-486b-9385-4a091c81d27e None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:45.187164 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b359a0-2c9b-486b-9385-4a091c81d27e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:45.188032 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b359a0-2c9b-486b-9385-4a091c81d27e None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:45.190120 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 136/269] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:02:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:02:45.192067 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f437b8-0ca2-49b0-856b-62a885731d89 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:45.192545 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f437b8-0ca2-49b0-856b-62a885731d89 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:45.193033 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f437b8-0ca2-49b0-856b-62a885731d89 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:45.195218 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 137/270] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:02:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:02:45.199450 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bed8f1b-8427-4651-99d2-713d13b08e49 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:45.199450 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bed8f1b-8427-4651-99d2-713d13b08e49 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:45.199450 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bed8f1b-8427-4651-99d2-713d13b08e49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:45.205923 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:45.216714 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 134/271] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 6 16:02:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 16:02:45.218255 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a261a-04c7-4cf8-b137-f73ea5962e05 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:45.218674 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-0bed8f1b-8427-4651-99d2-713d13b08e49 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:45.218970 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a261a-04c7-4cf8-b137-f73ea5962e05 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:45.219430 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a261a-04c7-4cf8-b137-f73ea5962e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:45.219864 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0bed8f1b-8427-4651-99d2-713d13b08e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:45.237707 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-436a261a-04c7-4cf8-b137-f73ea5962e05 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:45.239238 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-436a261a-04c7-4cf8-b137-f73ea5962e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:45.285762 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:45.292304 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 138/272] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:02:45.293824 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bed8f1b-8427-4651-99d2-713d13b08e49 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:45.320486 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 135/273] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:02:45.398664 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bed8f1b-8427-4651-99d2-713d13b08e49 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:45.434251 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fc457c-6bd7-4438-bfdc-0e70690f7d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:45.435208 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fc457c-6bd7-4438-bfdc-0e70690f7d9d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:45.435685 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fc457c-6bd7-4438-bfdc-0e70690f7d9d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:45.436157 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fc457c-6bd7-4438-bfdc-0e70690f7d9d None admin] PATH_INFO: `/v3/projects/cf9744296b5341f08017259b44d86f94` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:45.437519 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fc457c-6bd7-4438-bfdc-0e70690f7d9d None admin] RBAC: Authorizing `identity:get_project(project_id=cf9744296b5341f08017259b44d86f94)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:45.439242 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fc457c-6bd7-4438-bfdc-0e70690f7d9d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:45.441119 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 139/274] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 6 16:02:45 2022] GET /identity/v3/projects/cf9744296b5341f08017259b44d86f94 => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:45.447377 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436a261a-04c7-4cf8-b137-f73ea5962e05 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:45.583091 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436a261a-04c7-4cf8-b137-f73ea5962e05 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:45.598195 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fc457c-6bd7-4438-bfdc-0e70690f7d9d None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:45.615172 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:45.620499 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ed9b29-fa8a-46f2-9d80-9338cfca7596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:45.621156 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ed9b29-fa8a-46f2-9d80-9338cfca7596 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:45.621499 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ed9b29-fa8a-46f2-9d80-9338cfca7596 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:45.621689 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ed9b29-fa8a-46f2-9d80-9338cfca7596 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:45.622422 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ed9b29-fa8a-46f2-9d80-9338cfca7596 None admin] RBAC: Adding query filter params (role.id=61df19b017494bdcadc9a5e9d8bb9e52, scope.project.id=cf9744296b5341f08017259b44d86f94, user.id=a73439da3d3a43ca95973b14c17f910f) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:45.622726 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ed9b29-fa8a-46f2-9d80-9338cfca7596 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:45.624199 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ed9b29-fa8a-46f2-9d80-9338cfca7596 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:45.630285 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 136/275] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 6 16:02:45 2022] GET /identity/v3/role_assignments?user.id=a73439da3d3a43ca95973b14c17f910f&scope.project.id=cf9744296b5341f08017259b44d86f94&role.id=61df19b017494bdcadc9a5e9d8bb9e52 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:45.631866 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc302fa0-4a07-48b8-89d7-5704e54cc4db None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:45.632148 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc302fa0-4a07-48b8-89d7-5704e54cc4db None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:45.632418 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc302fa0-4a07-48b8-89d7-5704e54cc4db None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:45.633641 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 137/276] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:02:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:02:45.635630 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc302fa0-4a07-48b8-89d7-5704e54cc4db None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:45.698506 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:45.739232 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fc457c-6bd7-4438-bfdc-0e70690f7d9d None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:45.754293 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc302fa0-4a07-48b8-89d7-5704e54cc4db None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:45.777522 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5755dde5-5efe-45c2-ace9-df234ff61e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:45.778454 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5755dde5-5efe-45c2-ace9-df234ff61e4d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:45.778454 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5755dde5-5efe-45c2-ace9-df234ff61e4d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:45.778665 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5755dde5-5efe-45c2-ace9-df234ff61e4d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:45.789901 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5755dde5-5efe-45c2-ace9-df234ff61e4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:02:45.795344 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-5755dde5-5efe-45c2-ace9-df234ff61e4d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 16:02:45.795344 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:45.795344 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:45.795344 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:45.795344 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:45.795344 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:45.795344 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:45.795344 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:45.795344 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:45.795344 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:45.795344 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:45.795344 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:45.795344 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:02:45.795344 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:02:45.795344 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:02:45.795344 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:02:45.796262 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:02:45.796262 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:02:45.796262 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:02:45.796262 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:02:45.796262 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:45.796262 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:45.796262 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:45.796262 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:45.796262 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:45.796262 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:45.796262 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:45.796262 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:45.796262 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:45.796262 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:45.796262 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:45.796262 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:45.797149 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:45.797149 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:45.797149 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:45.797149 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:45.797149 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:02:45.797149 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:02:45.797149 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:02:45.797149 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:02:45.797149 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:02:45.797149 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:02:45.797149 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 16:02:45.797149 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:02:45.797743 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4888f8f-6261-4430-8925-d2e88ad01ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:45.797837 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 140/277] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 6 16:02:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:02:45.797905 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4888f8f-6261-4430-8925-d2e88ad01ccd None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:45.797905 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4888f8f-6261-4430-8925-d2e88ad01ccd None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:45.797905 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4888f8f-6261-4430-8925-d2e88ad01ccd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:45.802599 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3ff113-a16d-40a0-a4ad-3223930118f9 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:45.802599 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3ff113-a16d-40a0-a4ad-3223930118f9 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:45.802801 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3ff113-a16d-40a0-a4ad-3223930118f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:45.808712 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4888f8f-6261-4430-8925-d2e88ad01ccd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:02:45.813704 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-d4888f8f-6261-4430-8925-d2e88ad01ccd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 16:02:45.813704 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:45.813704 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:45.813704 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:45.813704 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:45.813704 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:45.813704 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:45.813704 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:45.813704 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:45.813704 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:45.813704 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:45.813704 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:45.813704 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:02:45.813704 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:02:45.813704 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:02:45.813704 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:02:45.814582 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:02:45.814582 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:02:45.814582 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:02:45.814582 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:02:45.814582 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:45.814582 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:45.814582 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:45.814582 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:45.814582 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:45.814582 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:45.814582 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:45.814582 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:45.814582 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:45.814582 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:45.814582 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:45.814582 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:45.815362 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:45.815362 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:45.815362 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:45.815362 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:45.815362 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:02:45.815362 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:02:45.815362 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:02:45.815362 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:02:45.815362 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:02:45.815362 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:02:45.815362 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 16:02:45.815362 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:02:45.815362 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 138/278] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 6 16:02:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:02:45.816010 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4888f8f-6261-4430-8925-d2e88ad01ccd None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:45.825452 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-7a3ff113-a16d-40a0-a4ad-3223930118f9 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:45.826633 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-7a3ff113-a16d-40a0-a4ad-3223930118f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:45.871690 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:45.897713 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 141/279] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:02:45.898583 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3ff113-a16d-40a0-a4ad-3223930118f9 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:45.926058 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4888f8f-6261-4430-8925-d2e88ad01ccd None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:45.957978 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:45.962019 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047d61b9-652c-4ab4-bf1d-5d9805b9ab99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:45.962661 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047d61b9-652c-4ab4-bf1d-5d9805b9ab99 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:45.962936 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047d61b9-652c-4ab4-bf1d-5d9805b9ab99 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:45.963203 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047d61b9-652c-4ab4-bf1d-5d9805b9ab99 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:45.964293 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047d61b9-652c-4ab4-bf1d-5d9805b9ab99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:45.965783 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047d61b9-652c-4ab4-bf1d-5d9805b9ab99 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:45.971841 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 139/280] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:02:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:45.971841 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047d61b9-652c-4ab4-bf1d-5d9805b9ab99 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:46.010224 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3ff113-a16d-40a0-a4ad-3223930118f9 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:46.045620 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326b5f1e-f904-4fdf-995a-5e72ca49b90a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:46.046317 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b5f1e-f904-4fdf-995a-5e72ca49b90a None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:46.046597 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b5f1e-f904-4fdf-995a-5e72ca49b90a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:46.046861 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b5f1e-f904-4fdf-995a-5e72ca49b90a None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:46.047524 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326b5f1e-f904-4fdf-995a-5e72ca49b90a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:46.047818 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326b5f1e-f904-4fdf-995a-5e72ca49b90a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:46.049362 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326b5f1e-f904-4fdf-995a-5e72ca49b90a None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:46.060667 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 142/281] 10.222.0.43 () {56 vars in 1101 bytes} [Tue Sep 6 16:02:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:46.061477 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326b5f1e-f904-4fdf-995a-5e72ca49b90a None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:46.083471 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047d61b9-652c-4ab4-bf1d-5d9805b9ab99 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:46.101706 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:46.122438 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f1ab26-c2ec-45f7-a32d-688cecb61d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:46.122983 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f1ab26-c2ec-45f7-a32d-688cecb61d1a None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:46.123281 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f1ab26-c2ec-45f7-a32d-688cecb61d1a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:46.123525 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f1ab26-c2ec-45f7-a32d-688cecb61d1a None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:46.124192 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f1ab26-c2ec-45f7-a32d-688cecb61d1a None admin] RBAC: Adding query filter params (name=member) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:46.124490 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f1ab26-c2ec-45f7-a32d-688cecb61d1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:46.125961 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f1ab26-c2ec-45f7-a32d-688cecb61d1a None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:46.139905 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 140/282] 10.222.0.43 () {56 vars in 1101 bytes} [Tue Sep 6 16:02:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:46.139905 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f1ab26-c2ec-45f7-a32d-688cecb61d1a None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:46.165318 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326b5f1e-f904-4fdf-995a-5e72ca49b90a None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:46.182116 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:46.198508 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8141e819-a5c1-4ef2-bef6-81a9a7a4fa4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:46.199110 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8141e819-a5c1-4ef2-bef6-81a9a7a4fa4d None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:46.199381 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8141e819-a5c1-4ef2-bef6-81a9a7a4fa4d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:46.199642 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8141e819-a5c1-4ef2-bef6-81a9a7a4fa4d None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:46.200284 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8141e819-a5c1-4ef2-bef6-81a9a7a4fa4d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:46.201774 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8141e819-a5c1-4ef2-bef6-81a9a7a4fa4d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:46.223150 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 143/283] 10.222.0.43 () {60 vars in 1132 bytes} [Tue Sep 6 16:02:46 2022] POST /identity/v3/users => generated 293 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:02:46.224378 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8141e819-a5c1-4ef2-bef6-81a9a7a4fa4d None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:46.259933 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f1ab26-c2ec-45f7-a32d-688cecb61d1a None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:46.301101 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c98c29-c1c2-4857-9571-028057d62595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:46.301771 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c98c29-c1c2-4857-9571-028057d62595 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:46.302020 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c98c29-c1c2-4857-9571-028057d62595 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:46.302288 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c98c29-c1c2-4857-9571-028057d62595 None admin] PATH_INFO: `/v3/users/c44807cbc8c24276b7a6496957d3330c` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:46.303565 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c98c29-c1c2-4857-9571-028057d62595 None admin] RBAC: Authorizing `identity:get_user(user_id=c44807cbc8c24276b7a6496957d3330c)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:46.305077 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c98c29-c1c2-4857-9571-028057d62595 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:46.306741 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 141/284] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 6 16:02:46 2022] GET /identity/v3/users/c44807cbc8c24276b7a6496957d3330c => generated 297 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:46.307515 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c98c29-c1c2-4857-9571-028057d62595 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:46.351166 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8141e819-a5c1-4ef2-bef6-81a9a7a4fa4d None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:46.361608 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:46.390836 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0865391c-7cc9-426d-b76b-54812cbe96af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:46.391399 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0865391c-7cc9-426d-b76b-54812cbe96af None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:46.391674 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0865391c-7cc9-426d-b76b-54812cbe96af None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:46.391946 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0865391c-7cc9-426d-b76b-54812cbe96af None admin] PATH_INFO: `/v3/users/89d065bf2c014f3aae12126c6c5e2638` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:46.413014 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0865391c-7cc9-426d-b76b-54812cbe96af None admin] RBAC: Authorizing `identity:get_user(user_id=89d065bf2c014f3aae12126c6c5e2638)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:46.414556 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0865391c-7cc9-426d-b76b-54812cbe96af None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:46.416322 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 144/285] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 6 16:02:46 2022] GET /identity/v3/users/89d065bf2c014f3aae12126c6c5e2638 => generated 293 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:46.417019 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0865391c-7cc9-426d-b76b-54812cbe96af None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:46.429729 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c98c29-c1c2-4857-9571-028057d62595 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:46.455937 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:46.468491 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04f195a-b5d3-4e3f-a099-13fe7b5b4e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:46.470295 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f195a-b5d3-4e3f-a099-13fe7b5b4e17 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:46.470295 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f195a-b5d3-4e3f-a099-13fe7b5b4e17 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:46.470295 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f195a-b5d3-4e3f-a099-13fe7b5b4e17 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:46.485159 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04f195a-b5d3-4e3f-a099-13fe7b5b4e17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:02:46.491354 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-f04f195a-b5d3-4e3f-a099-13fe7b5b4e17 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 16:02:46.491354 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:46.491354 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:46.491354 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:46.491354 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:46.491354 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:46.491354 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:46.491354 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:46.491354 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:46.491354 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:46.491354 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:46.491354 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:46.491354 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:02:46.491354 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:02:46.491354 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:02:46.491354 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:02:46.492297 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:02:46.492297 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:02:46.492297 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:02:46.492297 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:02:46.492297 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:46.492297 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:46.492297 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:46.492297 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:46.492297 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:46.492297 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:46.492297 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:46.492297 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:46.492297 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:46.492297 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:46.492297 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:46.492297 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:46.495178 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:46.495178 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:46.495178 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:46.495178 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:46.495178 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:02:46.495178 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:02:46.495178 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:02:46.495178 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:02:46.495178 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:02:46.495178 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:02:46.495178 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 16:02:46.495178 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:02:46.495178 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 142/286] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 6 16:02:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:02:46.495178 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04f195a-b5d3-4e3f-a099-13fe7b5b4e17 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:46.546951 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0865391c-7cc9-426d-b76b-54812cbe96af None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:46.602810 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c38cd1-dd8b-4dd9-9793-4563c4772f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:46.603713 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c38cd1-dd8b-4dd9-9793-4563c4772f7a None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:46.603987 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c38cd1-dd8b-4dd9-9793-4563c4772f7a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:46.604494 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c38cd1-dd8b-4dd9-9793-4563c4772f7a None admin] PATH_INFO: `/v3/projects/cf9744296b5341f08017259b44d86f94` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:46.606086 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c38cd1-dd8b-4dd9-9793-4563c4772f7a None admin] RBAC: Authorizing `identity:get_project(project_id=cf9744296b5341f08017259b44d86f94)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:46.608078 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c38cd1-dd8b-4dd9-9793-4563c4772f7a None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:46.610648 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 145/287] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 6 16:02:46 2022] GET /identity/v3/projects/cf9744296b5341f08017259b44d86f94 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:46.611790 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c38cd1-dd8b-4dd9-9793-4563c4772f7a None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:46.619829 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04f195a-b5d3-4e3f-a099-13fe7b5b4e17 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:46.627713 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:46.657043 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26741494-8253-48ac-8c2a-3729c9656a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:46.657568 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26741494-8253-48ac-8c2a-3729c9656a11 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:46.657893 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26741494-8253-48ac-8c2a-3729c9656a11 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:46.658163 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26741494-8253-48ac-8c2a-3729c9656a11 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:46.658896 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26741494-8253-48ac-8c2a-3729c9656a11 None admin] RBAC: Adding query filter params (role.id=15252b0b89894c1e8a6bd9a98b1c3ab7, user.id=89d065bf2c014f3aae12126c6c5e2638, scope.system=all) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:46.659211 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26741494-8253-48ac-8c2a-3729c9656a11 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:46.660739 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26741494-8253-48ac-8c2a-3729c9656a11 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:46.666851 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 143/288] 10.222.0.43 () {56 vars in 1308 bytes} [Tue Sep 6 16:02:46 2022] GET /identity/v3/role_assignments?user.id=89d065bf2c014f3aae12126c6c5e2638&scope.system=all&role.id=15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:46.667375 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26741494-8253-48ac-8c2a-3729c9656a11 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:46.730142 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:46.733665 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c38cd1-dd8b-4dd9-9793-4563c4772f7a None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:46.771656 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdda1974-789d-46a9-b40b-845f8204d193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:46.772909 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdda1974-789d-46a9-b40b-845f8204d193 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:46.773378 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdda1974-789d-46a9-b40b-845f8204d193 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:46.773834 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdda1974-789d-46a9-b40b-845f8204d193 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:46.774709 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdda1974-789d-46a9-b40b-845f8204d193 None admin] RBAC: Adding query filter params (name=member) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:46.775193 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdda1974-789d-46a9-b40b-845f8204d193 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:46.776792 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdda1974-789d-46a9-b40b-845f8204d193 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:46.786090 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 146/289] 10.222.0.43 () {56 vars in 1101 bytes} [Tue Sep 6 16:02:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:46.791955 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdda1974-789d-46a9-b40b-845f8204d193 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:46.854057 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26741494-8253-48ac-8c2a-3729c9656a11 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:46.930414 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e7a0fa-4bb6-4b29-8884-eb379fa250b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:46.931027 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e7a0fa-4bb6-4b29-8884-eb379fa250b5 None admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:46.931297 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e7a0fa-4bb6-4b29-8884-eb379fa250b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:46.931568 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e7a0fa-4bb6-4b29-8884-eb379fa250b5 None admin] PATH_INFO: `/v3/projects/cf9744296b5341f08017259b44d86f94/users/c44807cbc8c24276b7a6496957d3330c/roles/4908d6a02c4049e88249e9afbf7e2e9f` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:46.934876 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e7a0fa-4bb6-4b29-8884-eb379fa250b5 None admin] RBAC: Authorizing `identity:create_grant(project_id=cf9744296b5341f08017259b44d86f94, user_id=c44807cbc8c24276b7a6496957d3330c, role_id=4908d6a02c4049e88249e9afbf7e2e9f)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:46.936341 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e7a0fa-4bb6-4b29-8884-eb379fa250b5 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:46.947631 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdda1974-789d-46a9-b40b-845f8204d193 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:46.952849 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 144/290] 10.222.0.43 () {58 vars in 1439 bytes} [Tue Sep 6 16:02:46 2022] PUT /identity/v3/projects/cf9744296b5341f08017259b44d86f94/users/c44807cbc8c24276b7a6496957d3330c/roles/4908d6a02c4049e88249e9afbf7e2e9f => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:02:47.000247 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544e70e3-405b-4d6d-a455-eecec48ec716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:47.000942 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e70e3-405b-4d6d-a455-eecec48ec716 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:47.001222 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e70e3-405b-4d6d-a455-eecec48ec716 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:47.001497 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e70e3-405b-4d6d-a455-eecec48ec716 None admin] PATH_INFO: `/v3/users/74003d65e29b414f8fe73e69d5b3ce4b` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:47.002660 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544e70e3-405b-4d6d-a455-eecec48ec716 None admin] RBAC: Authorizing `identity:get_user(user_id=74003d65e29b414f8fe73e69d5b3ce4b)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:47.004147 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544e70e3-405b-4d6d-a455-eecec48ec716 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:47.006227 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 147/291] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 6 16:02:46 2022] GET /identity/v3/users/74003d65e29b414f8fe73e69d5b3ce4b => generated 297 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:02:47.012121 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e7a0fa-4bb6-4b29-8884-eb379fa250b5 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:47.061633 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:47.157511 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e7a0fa-4bb6-4b29-8884-eb379fa250b5 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:47.205943 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eec7e69-e169-4489-a22b-b7f2aaa1ffa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:47.206851 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eec7e69-e169-4489-a22b-b7f2aaa1ffa1 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:47.207136 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eec7e69-e169-4489-a22b-b7f2aaa1ffa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:47.207414 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eec7e69-e169-4489-a22b-b7f2aaa1ffa1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:47.303747 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eec7e69-e169-4489-a22b-b7f2aaa1ffa1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:47.305342 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eec7e69-e169-4489-a22b-b7f2aaa1ffa1 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:47.355307 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:47.440655 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 145/292] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 6 16:02:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 16:02:47.446003 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544e70e3-405b-4d6d-a455-eecec48ec716 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:47.492719 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:47.594863 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544e70e3-405b-4d6d-a455-eecec48ec716 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:47.636136 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd480a4-a5fe-4fe2-a1dc-26e45cecb6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:47.636799 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd480a4-a5fe-4fe2-a1dc-26e45cecb6a0 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:47.637085 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd480a4-a5fe-4fe2-a1dc-26e45cecb6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:47.637358 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd480a4-a5fe-4fe2-a1dc-26e45cecb6a0 None admin] PATH_INFO: `/v3/projects/cf9744296b5341f08017259b44d86f94` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:47.638579 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd480a4-a5fe-4fe2-a1dc-26e45cecb6a0 None admin] RBAC: Authorizing `identity:get_project(project_id=cf9744296b5341f08017259b44d86f94)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:47.640071 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd480a4-a5fe-4fe2-a1dc-26e45cecb6a0 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:47.641773 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 148/293] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 6 16:02:47 2022] GET /identity/v3/projects/cf9744296b5341f08017259b44d86f94 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:47.647833 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eec7e69-e169-4489-a22b-b7f2aaa1ffa1 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:47.796886 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eec7e69-e169-4489-a22b-b7f2aaa1ffa1 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:47.828144 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:47.834947 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902c3c00-b1dc-4e60-b26c-6f375386d854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:47.835562 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c3c00-b1dc-4e60-b26c-6f375386d854 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:47.835838 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c3c00-b1dc-4e60-b26c-6f375386d854 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:47.836106 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c3c00-b1dc-4e60-b26c-6f375386d854 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:47.836868 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902c3c00-b1dc-4e60-b26c-6f375386d854 None admin] RBAC: Adding query filter params (role.id=15252b0b89894c1e8a6bd9a98b1c3ab7, scope.project.id=cf9744296b5341f08017259b44d86f94, user.id=74003d65e29b414f8fe73e69d5b3ce4b) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:47.837175 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902c3c00-b1dc-4e60-b26c-6f375386d854 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:47.838614 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902c3c00-b1dc-4e60-b26c-6f375386d854 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:47.844781 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 146/294] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 6 16:02:47 2022] GET /identity/v3/role_assignments?user.id=74003d65e29b414f8fe73e69d5b3ce4b&scope.project.id=cf9744296b5341f08017259b44d86f94&role.id=15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:48.468728 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fc065a-6e53-4f02-b144-eeb09a82d9ce None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:48.469251 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fc065a-6e53-4f02-b144-eeb09a82d9ce None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:48.469324 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fc065a-6e53-4f02-b144-eeb09a82d9ce None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:48.470947 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 149/295] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:02:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:02:48.476111 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52de601-346a-4c8a-9912-ac084924b674 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:48.476398 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52de601-346a-4c8a-9912-ac084924b674 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:48.476673 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52de601-346a-4c8a-9912-ac084924b674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:48.496700 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-e52de601-346a-4c8a-9912-ac084924b674 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:48.497941 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e52de601-346a-4c8a-9912-ac084924b674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:48.566334 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 147/296] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:02:48.580872 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3513c1-a9de-47f3-9e50-a60b89bcd192 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:48.580872 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3513c1-a9de-47f3-9e50-a60b89bcd192 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:48.580872 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3513c1-a9de-47f3-9e50-a60b89bcd192 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:48.581876 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 150/297] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:02:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:02:48.587197 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a241622-b947-4378-97d8-9dcfc65291a4 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:48.587699 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a241622-b947-4378-97d8-9dcfc65291a4 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:48.588154 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a241622-b947-4378-97d8-9dcfc65291a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:48.606396 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-3a241622-b947-4378-97d8-9dcfc65291a4 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:48.607849 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-3a241622-b947-4378-97d8-9dcfc65291a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:48.678210 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 148/298] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:02:48.843709 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3513c1-a9de-47f3-9e50-a60b89bcd192 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:48.918871 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7503cb21-0fcb-433b-b5da-7b68aab3562b None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:48.919196 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7503cb21-0fcb-433b-b5da-7b68aab3562b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:48.919430 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7503cb21-0fcb-433b-b5da-7b68aab3562b None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:48.920819 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 149/299] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:02:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 16:02:48.926026 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b846981-c683-4883-8b94-f1b247d15a55 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:48.926311 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b846981-c683-4883-8b94-f1b247d15a55 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:48.926585 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b846981-c683-4883-8b94-f1b247d15a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:48.947129 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:48.950489 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-3b846981-c683-4883-8b94-f1b247d15a55 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:48.951704 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-3b846981-c683-4883-8b94-f1b247d15a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:48.952192 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3513c1-a9de-47f3-9e50-a60b89bcd192 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:48.987280 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d087e34-f421-4c73-9048-5746ba19f454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:48.987918 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d087e34-f421-4c73-9048-5746ba19f454 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:48.988199 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d087e34-f421-4c73-9048-5746ba19f454 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:48.988474 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d087e34-f421-4c73-9048-5746ba19f454 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:48.999486 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d087e34-f421-4c73-9048-5746ba19f454 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:02:49.004320 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-8d087e34-f421-4c73-9048-5746ba19f454 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 16:02:49.004320 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:49.004320 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:49.004320 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:49.004320 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:49.004320 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:49.004320 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:49.004320 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:49.004320 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:49.004320 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:49.004320 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:49.004320 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:49.004320 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:02:49.004320 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:02:49.004320 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:02:49.004320 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:02:49.005442 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:02:49.005442 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:02:49.005442 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:02:49.005442 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:02:49.005442 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:49.005442 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:49.005442 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:49.005442 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:49.005442 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:49.005442 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:49.005442 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:49.005442 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:49.005442 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:49.005442 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:49.005442 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:49.005442 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:49.006280 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:49.006280 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:49.006280 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:49.006280 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:49.006280 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:02:49.006280 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:02:49.006280 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:02:49.006280 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:02:49.006280 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:02:49.006280 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:02:49.006280 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 16:02:49.006280 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:02:49.006280 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 151/300] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 6 16:02:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:02:49.006280 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d087e34-f421-4c73-9048-5746ba19f454 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:49.016706 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:49.045161 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 150/301] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 16:02:49.048359 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b846981-c683-4883-8b94-f1b247d15a55 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:49.128746 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d087e34-f421-4c73-9048-5746ba19f454 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:49.172437 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cf55b0-e987-498b-b135-54ae314d6975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:49.173115 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf55b0-e987-498b-b135-54ae314d6975 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:49.173388 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf55b0-e987-498b-b135-54ae314d6975 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:49.173657 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf55b0-e987-498b-b135-54ae314d6975 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:49.173756 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b846981-c683-4883-8b94-f1b247d15a55 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:49.174749 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cf55b0-e987-498b-b135-54ae314d6975 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:49.177420 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cf55b0-e987-498b-b135-54ae314d6975 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:49.184221 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 152/302] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:02:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:49.185093 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cf55b0-e987-498b-b135-54ae314d6975 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:49.212414 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713f9b4e-f1a7-4410-abe1-c2dfb74e3943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:49.213478 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713f9b4e-f1a7-4410-abe1-c2dfb74e3943 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:49.213961 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713f9b4e-f1a7-4410-abe1-c2dfb74e3943 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:49.214444 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713f9b4e-f1a7-4410-abe1-c2dfb74e3943 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:49.215300 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713f9b4e-f1a7-4410-abe1-c2dfb74e3943 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:49.215790 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713f9b4e-f1a7-4410-abe1-c2dfb74e3943 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:49.217529 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713f9b4e-f1a7-4410-abe1-c2dfb74e3943 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:49.221078 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:49.228613 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 151/303] 10.222.0.43 () {56 vars in 1101 bytes} [Tue Sep 6 16:02:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:49.234291 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713f9b4e-f1a7-4410-abe1-c2dfb74e3943 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:49.288257 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:49.325809 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cf55b0-e987-498b-b135-54ae314d6975 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:49.352131 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713f9b4e-f1a7-4410-abe1-c2dfb74e3943 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:49.360468 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced04cf8-717d-4e8d-9152-45ba38bb6b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:49.361380 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced04cf8-717d-4e8d-9152-45ba38bb6b93 None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:49.361655 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced04cf8-717d-4e8d-9152-45ba38bb6b93 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:49.361922 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced04cf8-717d-4e8d-9152-45ba38bb6b93 None admin] PATH_INFO: `/v3/groups` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:49.362600 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced04cf8-717d-4e8d-9152-45ba38bb6b93 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:49.364043 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced04cf8-717d-4e8d-9152-45ba38bb6b93 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:49.384876 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 153/304] 10.222.0.43 () {60 vars in 1134 bytes} [Tue Sep 6 16:02:49 2022] POST /identity/v3/groups => generated 229 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:02:49.384876 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced04cf8-717d-4e8d-9152-45ba38bb6b93 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:49.395931 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1185940-dc5c-40f7-b457-22f56611cefe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:49.396994 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1185940-dc5c-40f7-b457-22f56611cefe None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:49.397278 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1185940-dc5c-40f7-b457-22f56611cefe None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:49.397556 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1185940-dc5c-40f7-b457-22f56611cefe None admin] PATH_INFO: `/v3/users/f1ebbcd3c6be44178b34f78d12b202a2` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:49.413396 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1185940-dc5c-40f7-b457-22f56611cefe None admin] RBAC: Authorizing `identity:get_user(user_id=f1ebbcd3c6be44178b34f78d12b202a2)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:49.414959 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1185940-dc5c-40f7-b457-22f56611cefe None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:49.417076 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 152/305] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 6 16:02:49 2022] GET /identity/v3/users/f1ebbcd3c6be44178b34f78d12b202a2 => generated 293 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:02:49.418248 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ea237d-eade-4a01-a66c-437c2868b46e None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:49.418534 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ea237d-eade-4a01-a66c-437c2868b46e None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:49.418810 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ea237d-eade-4a01-a66c-437c2868b46e None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:49.420249 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 153/306] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:02:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:02:49.425096 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ea237d-eade-4a01-a66c-437c2868b46e None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:49.487182 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:49.505859 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced04cf8-717d-4e8d-9152-45ba38bb6b93 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:49.541959 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ea237d-eade-4a01-a66c-437c2868b46e None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:49.544374 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49a7925-6503-4628-a9ff-11398fe4cb5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:49.545027 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:49.545142 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a7925-6503-4628-a9ff-11398fe4cb5d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:49.545305 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a7925-6503-4628-a9ff-11398fe4cb5d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:49.545598 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a7925-6503-4628-a9ff-11398fe4cb5d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:49.558750 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49a7925-6503-4628-a9ff-11398fe4cb5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:02:49.563841 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-e49a7925-6503-4628-a9ff-11398fe4cb5d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 16:02:49.563841 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:49.563841 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:49.563841 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:49.563841 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:49.563841 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:49.563841 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:49.563841 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:49.563841 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:49.563841 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:49.563841 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:49.563841 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:49.563841 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:02:49.563841 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:02:49.563841 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:02:49.563841 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:02:49.564746 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:02:49.564746 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:02:49.564746 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:02:49.564746 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:02:49.564746 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:49.564746 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:49.564746 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:49.564746 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:49.564746 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:49.564746 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:49.564746 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:49.564746 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:49.564746 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:49.564746 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:49.564746 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:49.564746 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:49.565613 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:49.565613 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:49.565613 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:49.565613 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:49.565613 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:02:49.565613 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:02:49.565613 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:02:49.565613 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:02:49.565613 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:02:49.565613 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:02:49.565613 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 16:02:49.565613 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:02:49.565613 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 154/307] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 6 16:02:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:02:49.566270 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e88513-b81f-42ef-822e-319c3b20a094 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:49.567058 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e88513-b81f-42ef-822e-319c3b20a094 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:49.567058 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e88513-b81f-42ef-822e-319c3b20a094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:49.584297 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34698f52-c8e8-4a92-a737-e6365203eb99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:49.584946 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34698f52-c8e8-4a92-a737-e6365203eb99 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:49.585226 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34698f52-c8e8-4a92-a737-e6365203eb99 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:49.585499 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34698f52-c8e8-4a92-a737-e6365203eb99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:49.586224 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34698f52-c8e8-4a92-a737-e6365203eb99 None admin] RBAC: Adding query filter params (role.id=4908d6a02c4049e88249e9afbf7e2e9f, user.id=f1ebbcd3c6be44178b34f78d12b202a2, scope.system=all) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:49.586530 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34698f52-c8e8-4a92-a737-e6365203eb99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:49.586729 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-b3e88513-b81f-42ef-822e-319c3b20a094 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:49.587915 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b3e88513-b81f-42ef-822e-319c3b20a094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:49.588009 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34698f52-c8e8-4a92-a737-e6365203eb99 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:49.597257 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 154/308] 10.222.0.43 () {56 vars in 1308 bytes} [Tue Sep 6 16:02:49 2022] GET /identity/v3/role_assignments?user.id=f1ebbcd3c6be44178b34f78d12b202a2&scope.system=all&role.id=4908d6a02c4049e88249e9afbf7e2e9f => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:49.597257 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34698f52-c8e8-4a92-a737-e6365203eb99 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:49.674223 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 155/309] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:02:49.743272 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34698f52-c8e8-4a92-a737-e6365203eb99 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:49.783415 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b118c642-0433-4303-ad18-96d46380036b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:49.784044 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b118c642-0433-4303-ad18-96d46380036b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:49.784322 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b118c642-0433-4303-ad18-96d46380036b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:49.784590 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b118c642-0433-4303-ad18-96d46380036b None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:49.785293 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b118c642-0433-4303-ad18-96d46380036b None admin] RBAC: Adding query filter params (name=member) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:49.785593 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b118c642-0433-4303-ad18-96d46380036b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:49.786981 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b118c642-0433-4303-ad18-96d46380036b None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:49.796503 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 155/310] 10.222.0.43 () {56 vars in 1101 bytes} [Tue Sep 6 16:02:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:49.801913 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e88513-b81f-42ef-822e-319c3b20a094 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:49.879881 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:49.909208 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e88513-b81f-42ef-822e-319c3b20a094 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:49.944503 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b118c642-0433-4303-ad18-96d46380036b None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:49.947873 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b1c04b-1ccb-497e-947e-a82b84642ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:49.948845 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b1c04b-1ccb-497e-947e-a82b84642ac5 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:49.949133 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b1c04b-1ccb-497e-947e-a82b84642ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:49.949400 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b1c04b-1ccb-497e-947e-a82b84642ac5 None admin] PATH_INFO: `/v3/users/89d065bf2c014f3aae12126c6c5e2638` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:49.950588 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b1c04b-1ccb-497e-947e-a82b84642ac5 None admin] RBAC: Authorizing `identity:get_user(user_id=89d065bf2c014f3aae12126c6c5e2638)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:49.952056 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b1c04b-1ccb-497e-947e-a82b84642ac5 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:49.954002 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 156/311] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 6 16:02:49 2022] GET /identity/v3/users/89d065bf2c014f3aae12126c6c5e2638 => generated 293 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:02:49.961467 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b1c04b-1ccb-497e-947e-a82b84642ac5 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:49.987912 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:50.069403 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b118c642-0433-4303-ad18-96d46380036b None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:50.075572 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b1c04b-1ccb-497e-947e-a82b84642ac5 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:50.107892 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5d7486-450e-43c2-8669-237b091b42c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:50.108921 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5d7486-450e-43c2-8669-237b091b42c5 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:50.109413 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5d7486-450e-43c2-8669-237b091b42c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:50.109874 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5d7486-450e-43c2-8669-237b091b42c5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:50.110953 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866c65fb-f781-4d1e-80f9-fda1894cec73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:50.111621 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c65fb-f781-4d1e-80f9-fda1894cec73 None admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:50.111898 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c65fb-f781-4d1e-80f9-fda1894cec73 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:50.112170 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c65fb-f781-4d1e-80f9-fda1894cec73 None admin] PATH_INFO: `/v3/system/users/89d065bf2c014f3aae12126c6c5e2638/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:50.113804 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866c65fb-f781-4d1e-80f9-fda1894cec73 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=89d065bf2c014f3aae12126c6c5e2638, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:50.115317 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866c65fb-f781-4d1e-80f9-fda1894cec73 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:50.123302 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5d7486-450e-43c2-8669-237b091b42c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:02:50.128302 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-3e5d7486-450e-43c2-8669-237b091b42c5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 16:02:50.128302 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:50.128302 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:50.128302 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:50.128302 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:50.128302 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:50.128302 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:50.128302 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:50.128302 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:50.128302 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:50.128302 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:50.128302 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:50.128302 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:02:50.128302 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:02:50.128302 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:02:50.128302 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:02:50.129641 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:02:50.129641 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:02:50.129641 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:02:50.129641 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:02:50.129641 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:50.129641 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:50.129641 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:50.129641 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:50.129641 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:50.129641 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:50.129641 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:50.129641 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:50.129641 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:50.129641 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:50.129641 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:50.129641 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:50.130517 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:50.130517 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:50.130517 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:50.130517 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:50.130517 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:02:50.130517 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:02:50.130517 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:02:50.130517 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:02:50.130517 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:02:50.130517 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:02:50.130517 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 16:02:50.130517 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:02:50.130517 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 156/312] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 6 16:02:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:02:50.132341 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 157/313] 10.222.0.43 () {58 vars in 1334 bytes} [Tue Sep 6 16:02:49 2022] PUT /identity/v3/system/users/89d065bf2c014f3aae12126c6c5e2638/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:02:50.139450 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5d7486-450e-43c2-8669-237b091b42c5 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:50.245518 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:50.256386 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5d7486-450e-43c2-8669-237b091b42c5 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:50.294284 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aee87c3-1c9d-4afb-95c5-bed07cbe8460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:50.294882 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee87c3-1c9d-4afb-95c5-bed07cbe8460 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:50.295161 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee87c3-1c9d-4afb-95c5-bed07cbe8460 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:50.295490 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee87c3-1c9d-4afb-95c5-bed07cbe8460 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:50.296161 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aee87c3-1c9d-4afb-95c5-bed07cbe8460 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:50.296495 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aee87c3-1c9d-4afb-95c5-bed07cbe8460 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:50.297924 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aee87c3-1c9d-4afb-95c5-bed07cbe8460 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:50.306551 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 157/314] 10.222.0.43 () {56 vars in 1101 bytes} [Tue Sep 6 16:02:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:50.312449 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866c65fb-f781-4d1e-80f9-fda1894cec73 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:50.327565 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:50.419924 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866c65fb-f781-4d1e-80f9-fda1894cec73 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:50.454615 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948f52fd-d17a-411e-aa68-77b2ddf55a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:50.455331 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f52fd-d17a-411e-aa68-77b2ddf55a84 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:50.455613 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f52fd-d17a-411e-aa68-77b2ddf55a84 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:50.455882 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f52fd-d17a-411e-aa68-77b2ddf55a84 None admin] PATH_INFO: `/v3/users/c44807cbc8c24276b7a6496957d3330c` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:50.457425 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948f52fd-d17a-411e-aa68-77b2ddf55a84 None admin] RBAC: Authorizing `identity:get_user(user_id=c44807cbc8c24276b7a6496957d3330c)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:50.458898 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948f52fd-d17a-411e-aa68-77b2ddf55a84 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:50.460990 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 158/315] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 6 16:02:50 2022] GET /identity/v3/users/c44807cbc8c24276b7a6496957d3330c => generated 297 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:50.466678 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aee87c3-1c9d-4afb-95c5-bed07cbe8460 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:50.575031 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aee87c3-1c9d-4afb-95c5-bed07cbe8460 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:50.611087 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fe199e-1285-4cae-bc58-24387b9b383e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:50.612109 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe199e-1285-4cae-bc58-24387b9b383e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:50.612580 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe199e-1285-4cae-bc58-24387b9b383e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:50.613079 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe199e-1285-4cae-bc58-24387b9b383e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:50.635946 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:50.684956 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fe199e-1285-4cae-bc58-24387b9b383e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:50.686393 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fe199e-1285-4cae-bc58-24387b9b383e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:50.795553 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 158/316] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 6 16:02:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 16:02:50.805867 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948f52fd-d17a-411e-aa68-77b2ddf55a84 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:50.885265 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:50.907543 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948f52fd-d17a-411e-aa68-77b2ddf55a84 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:50.941267 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b443056-643b-4738-a75a-ea758a132939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:50.941903 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b443056-643b-4738-a75a-ea758a132939 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:50.942175 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b443056-643b-4738-a75a-ea758a132939 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:50.942439 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b443056-643b-4738-a75a-ea758a132939 None admin] PATH_INFO: `/v3/projects/cf9744296b5341f08017259b44d86f94` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:50.943619 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b443056-643b-4738-a75a-ea758a132939 None admin] RBAC: Authorizing `identity:get_project(project_id=cf9744296b5341f08017259b44d86f94)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:50.945115 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b443056-643b-4738-a75a-ea758a132939 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:50.946961 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 159/317] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 6 16:02:50 2022] GET /identity/v3/projects/cf9744296b5341f08017259b44d86f94 => generated 302 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:50.952826 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fe199e-1285-4cae-bc58-24387b9b383e None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:51.005486 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:51.056996 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fe199e-1285-4cae-bc58-24387b9b383e None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:51.091719 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f242be8a-414d-4502-bd23-1d94e7dc4d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:51.092323 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f242be8a-414d-4502-bd23-1d94e7dc4d89 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:51.092598 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f242be8a-414d-4502-bd23-1d94e7dc4d89 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:51.092895 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f242be8a-414d-4502-bd23-1d94e7dc4d89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:51.093619 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f242be8a-414d-4502-bd23-1d94e7dc4d89 None admin] RBAC: Adding query filter params (role.id=4908d6a02c4049e88249e9afbf7e2e9f, scope.project.id=cf9744296b5341f08017259b44d86f94, user.id=c44807cbc8c24276b7a6496957d3330c) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:51.093928 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f242be8a-414d-4502-bd23-1d94e7dc4d89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:51.095373 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f242be8a-414d-4502-bd23-1d94e7dc4d89 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:51.101636 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 159/318] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 6 16:02:50 2022] GET /identity/v3/role_assignments?user.id=c44807cbc8c24276b7a6496957d3330c&scope.project.id=cf9744296b5341f08017259b44d86f94&role.id=4908d6a02c4049e88249e9afbf7e2e9f => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:51.441557 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6b3bb8-f603-4c54-950e-90134b32f8db None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:51.441991 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6b3bb8-f603-4c54-950e-90134b32f8db None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:51.442188 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6b3bb8-f603-4c54-950e-90134b32f8db None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:51.443510 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 160/319] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:02:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:02:51.448686 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e197350-17cb-4de3-aa3a-783c5b1394f5 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:51.449005 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e197350-17cb-4de3-aa3a-783c5b1394f5 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:51.449275 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e197350-17cb-4de3-aa3a-783c5b1394f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:51.472361 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-9e197350-17cb-4de3-aa3a-783c5b1394f5 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:51.475945 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-9e197350-17cb-4de3-aa3a-783c5b1394f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:51.534235 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af20fd3-da67-43e5-9395-718862ca874a None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:51.534415 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af20fd3-da67-43e5-9395-718862ca874a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:51.534685 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af20fd3-da67-43e5-9395-718862ca874a None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:51.536036 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 161/320] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:02:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 16:02:51.541052 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1197473f-9160-4075-b626-ab1eabc08e12 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:51.541321 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1197473f-9160-4075-b626-ab1eabc08e12 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:51.541624 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1197473f-9160-4075-b626-ab1eabc08e12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:51.552787 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 160/321] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:02:51.559956 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-1197473f-9160-4075-b626-ab1eabc08e12 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:51.561190 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1197473f-9160-4075-b626-ab1eabc08e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:51.630972 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 162/322] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 16:02:51.818264 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e197350-17cb-4de3-aa3a-783c5b1394f5 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:51.865389 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:51.897758 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1197473f-9160-4075-b626-ab1eabc08e12 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:51.920489 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e197350-17cb-4de3-aa3a-783c5b1394f5 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:51.956864 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e600fa-3894-4f9f-9866-f22d8191dc75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:51.957519 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e600fa-3894-4f9f-9866-f22d8191dc75 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:51.957800 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e600fa-3894-4f9f-9866-f22d8191dc75 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:51.958068 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e600fa-3894-4f9f-9866-f22d8191dc75 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:51.959241 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e600fa-3894-4f9f-9866-f22d8191dc75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:51.960730 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e600fa-3894-4f9f-9866-f22d8191dc75 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:51.962497 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 161/323] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:02:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:51.967699 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e600fa-3894-4f9f-9866-f22d8191dc75 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:51.976649 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:52.002268 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1197473f-9160-4075-b626-ab1eabc08e12 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:52.038528 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c12be94-c83f-4899-8cbc-e0c41509b1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:52.039165 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12be94-c83f-4899-8cbc-e0c41509b1b7 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:52.039448 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12be94-c83f-4899-8cbc-e0c41509b1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:52.039754 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12be94-c83f-4899-8cbc-e0c41509b1b7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:52.050265 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c12be94-c83f-4899-8cbc-e0c41509b1b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:02:52.055287 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-1c12be94-c83f-4899-8cbc-e0c41509b1b7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 16:02:52.055287 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:52.055287 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:52.055287 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:52.055287 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:52.055287 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:52.055287 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:52.055287 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:52.055287 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:52.055287 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:52.055287 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:52.055287 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:52.055287 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:02:52.055287 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:02:52.055287 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:02:52.055287 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:02:52.056366 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:02:52.056366 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:02:52.056366 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:02:52.056366 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:02:52.056366 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:52.056366 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:52.056366 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:52.056366 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:52.056366 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:52.056366 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:52.056366 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:52.056366 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:52.056366 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:52.056366 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:52.056366 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:52.056366 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:52.057358 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:52.057358 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:52.057358 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:52.057358 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:52.057358 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:02:52.057358 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:02:52.057358 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:02:52.057358 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:02:52.057358 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:02:52.057358 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:02:52.057358 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 16:02:52.057358 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:02:52.057358 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 163/324] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 6 16:02:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:02:52.062052 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c12be94-c83f-4899-8cbc-e0c41509b1b7 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:52.076934 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e600fa-3894-4f9f-9866-f22d8191dc75 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:52.092379 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:52.113422 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044ec81b-5080-4474-bb3b-1c7d2195cf49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:52.114110 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044ec81b-5080-4474-bb3b-1c7d2195cf49 None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:52.114385 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044ec81b-5080-4474-bb3b-1c7d2195cf49 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:52.114650 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044ec81b-5080-4474-bb3b-1c7d2195cf49 None admin] PATH_INFO: `/v3/groups` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:52.115274 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044ec81b-5080-4474-bb3b-1c7d2195cf49 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:52.116816 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044ec81b-5080-4474-bb3b-1c7d2195cf49 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:52.133624 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 162/325] 10.222.0.43 () {60 vars in 1134 bytes} [Tue Sep 6 16:02:51 2022] POST /identity/v3/groups => generated 226 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:02:52.134918 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200b7ab4-ae17-41b9-a8a0-2a2aebf7ce44 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:52.135260 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200b7ab4-ae17-41b9-a8a0-2a2aebf7ce44 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:52.135530 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200b7ab4-ae17-41b9-a8a0-2a2aebf7ce44 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:52.136924 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 163/326] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:02:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:02:52.153744 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeecc0ef-8ff7-49cd-be0f-7c84549edbf8 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:52.154089 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeecc0ef-8ff7-49cd-be0f-7c84549edbf8 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:52.154301 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeecc0ef-8ff7-49cd-be0f-7c84549edbf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:52.174764 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-aeecc0ef-8ff7-49cd-be0f-7c84549edbf8 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:52.175636 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c12be94-c83f-4899-8cbc-e0c41509b1b7 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:52.175997 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-aeecc0ef-8ff7-49cd-be0f-7c84549edbf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:52.215195 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bf183b-bf33-4188-8f3f-2b10cb90179f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:52.216120 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bf183b-bf33-4188-8f3f-2b10cb90179f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:52.217364 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bf183b-bf33-4188-8f3f-2b10cb90179f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:52.217672 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bf183b-bf33-4188-8f3f-2b10cb90179f None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:52.218367 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bf183b-bf33-4188-8f3f-2b10cb90179f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:52.218670 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bf183b-bf33-4188-8f3f-2b10cb90179f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:52.220112 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bf183b-bf33-4188-8f3f-2b10cb90179f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:52.222390 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:52.231132 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 164/327] 10.222.0.43 () {56 vars in 1101 bytes} [Tue Sep 6 16:02:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:02:52.236979 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bf183b-bf33-4188-8f3f-2b10cb90179f None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:52.268387 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 164/328] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:02:52.361098 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bf183b-bf33-4188-8f3f-2b10cb90179f None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:52.396193 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c01cb4-cc68-453b-bba1-b81222c98c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:52.396949 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c01cb4-cc68-453b-bba1-b81222c98c1e None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:52.397231 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c01cb4-cc68-453b-bba1-b81222c98c1e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:52.397503 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c01cb4-cc68-453b-bba1-b81222c98c1e None admin] PATH_INFO: `/v3/users/f1ebbcd3c6be44178b34f78d12b202a2` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:52.398642 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c01cb4-cc68-453b-bba1-b81222c98c1e None admin] RBAC: Authorizing `identity:get_user(user_id=f1ebbcd3c6be44178b34f78d12b202a2)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:52.400120 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c01cb4-cc68-453b-bba1-b81222c98c1e None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:52.401965 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 165/329] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 6 16:02:52 2022] GET /identity/v3/users/f1ebbcd3c6be44178b34f78d12b202a2 => generated 293 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:52.407729 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeecc0ef-8ff7-49cd-be0f-7c84549edbf8 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:52.515879 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeecc0ef-8ff7-49cd-be0f-7c84549edbf8 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:52.518792 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:52.528125 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c01cb4-cc68-453b-bba1-b81222c98c1e None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:52.562428 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2f8c91-32cf-4759-852f-687cc43ab932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:52.563451 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2f8c91-32cf-4759-852f-687cc43ab932 None admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:52.563906 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2f8c91-32cf-4759-852f-687cc43ab932 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:52.564350 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2f8c91-32cf-4759-852f-687cc43ab932 None admin] PATH_INFO: `/v3/system/users/f1ebbcd3c6be44178b34f78d12b202a2/roles/4908d6a02c4049e88249e9afbf7e2e9f` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:52.567492 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2f8c91-32cf-4759-852f-687cc43ab932 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=f1ebbcd3c6be44178b34f78d12b202a2, role_id=4908d6a02c4049e88249e9afbf7e2e9f)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:52.569211 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2f8c91-32cf-4759-852f-687cc43ab932 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:52.584861 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 165/330] 10.222.0.43 () {58 vars in 1334 bytes} [Tue Sep 6 16:02:52 2022] PUT /identity/v3/system/users/f1ebbcd3c6be44178b34f78d12b202a2/roles/4908d6a02c4049e88249e9afbf7e2e9f => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:02:52.617998 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:52.664606 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c01cb4-cc68-453b-bba1-b81222c98c1e None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:52.704828 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8413c912-c64e-442b-a9fa-61718b22441d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:52.705489 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8413c912-c64e-442b-a9fa-61718b22441d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:52.705823 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8413c912-c64e-442b-a9fa-61718b22441d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:52.706462 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8413c912-c64e-442b-a9fa-61718b22441d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:52.718928 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8413c912-c64e-442b-a9fa-61718b22441d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:02:52.724782 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-8413c912-c64e-442b-a9fa-61718b22441d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 16:02:52.724782 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:52.724782 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:52.724782 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:52.724782 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:52.724782 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:52.724782 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:52.724782 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:52.724782 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:52.724782 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:52.724782 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:52.724782 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:52.724782 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:02:52.724782 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:02:52.724782 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:02:52.724782 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:02:52.725727 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:02:52.725727 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:02:52.725727 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:02:52.725727 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:02:52.725727 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:52.725727 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:52.725727 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:52.725727 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:52.725727 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:52.725727 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:52.725727 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:52.725727 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:52.725727 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:52.725727 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:52.725727 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:52.725727 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:52.726524 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:52.726524 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:52.726524 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:52.726524 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:52.726524 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:02:52.726524 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:02:52.726524 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:02:52.726524 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:02:52.726524 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:02:52.726524 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:02:52.726524 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 16:02:52.726524 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:02:52.727117 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 166/331] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 6 16:02:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:02:52.732635 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2f8c91-32cf-4759-852f-687cc43ab932 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:52.857664 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2f8c91-32cf-4759-852f-687cc43ab932 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:52.894291 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64307c9-620f-493d-a9b4-0095dd1bf086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:52.894841 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64307c9-620f-493d-a9b4-0095dd1bf086 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:52.895117 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64307c9-620f-493d-a9b4-0095dd1bf086 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:52.895397 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64307c9-620f-493d-a9b4-0095dd1bf086 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:52.896130 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64307c9-620f-493d-a9b4-0095dd1bf086 None admin] RBAC: Adding query filter params (name=member) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:52.896471 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64307c9-620f-493d-a9b4-0095dd1bf086 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:52.897970 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64307c9-620f-493d-a9b4-0095dd1bf086 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:52.910749 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 166/332] 10.222.0.43 () {56 vars in 1101 bytes} [Tue Sep 6 16:02:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:52.916521 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8413c912-c64e-442b-a9fa-61718b22441d None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:53.023938 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8413c912-c64e-442b-a9fa-61718b22441d None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:53.048464 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:53.060910 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03a622d-c657-40a9-bd14-e4b3426e8147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:53.061554 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03a622d-c657-40a9-bd14-e4b3426e8147 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:53.061829 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03a622d-c657-40a9-bd14-e4b3426e8147 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:53.062094 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03a622d-c657-40a9-bd14-e4b3426e8147 None admin] PATH_INFO: `/v3/users/89d065bf2c014f3aae12126c6c5e2638` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:53.063243 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03a622d-c657-40a9-bd14-e4b3426e8147 None admin] RBAC: Authorizing `identity:get_user(user_id=89d065bf2c014f3aae12126c6c5e2638)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:53.064700 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03a622d-c657-40a9-bd14-e4b3426e8147 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:53.066333 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 167/333] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 6 16:02:52 2022] GET /identity/v3/users/89d065bf2c014f3aae12126c6c5e2638 => generated 293 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:53.072484 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64307c9-620f-493d-a9b4-0095dd1bf086 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:53.105389 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:53.182044 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64307c9-620f-493d-a9b4-0095dd1bf086 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:53.218836 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2febbb99-6480-4641-aef5-824f3b5a2d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:53.219448 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2febbb99-6480-4641-aef5-824f3b5a2d85 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:53.219724 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2febbb99-6480-4641-aef5-824f3b5a2d85 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:53.219994 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2febbb99-6480-4641-aef5-824f3b5a2d85 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:53.220723 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2febbb99-6480-4641-aef5-824f3b5a2d85 None admin] RBAC: Adding query filter params (role.id=15252b0b89894c1e8a6bd9a98b1c3ab7, user.id=89d065bf2c014f3aae12126c6c5e2638, scope.system=all) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:53.221060 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2febbb99-6480-4641-aef5-824f3b5a2d85 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:53.222576 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2febbb99-6480-4641-aef5-824f3b5a2d85 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:53.237156 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 167/334] 10.222.0.43 () {56 vars in 1308 bytes} [Tue Sep 6 16:02:53 2022] GET /identity/v3/role_assignments?user.id=89d065bf2c014f3aae12126c6c5e2638&scope.system=all&role.id=15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 510 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:54.362841 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c653aa7-9a03-430e-9976-6c8869af29b4 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:54.363361 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c653aa7-9a03-430e-9976-6c8869af29b4 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:54.363361 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c653aa7-9a03-430e-9976-6c8869af29b4 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:54.365053 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 168/335] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:02:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:02:54.370247 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43feca31-62f4-4fac-8db3-da432cb356c4 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:54.370698 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43feca31-62f4-4fac-8db3-da432cb356c4 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:54.370774 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43feca31-62f4-4fac-8db3-da432cb356c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:54.393532 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-43feca31-62f4-4fac-8db3-da432cb356c4 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:54.394744 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-43feca31-62f4-4fac-8db3-da432cb356c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:54.471049 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 168/336] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:02:54.769431 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c653aa7-9a03-430e-9976-6c8869af29b4 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:54.808289 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65adea07-563c-4fed-a64c-27db0a93e4be None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:54.808622 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65adea07-563c-4fed-a64c-27db0a93e4be None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:54.808872 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65adea07-563c-4fed-a64c-27db0a93e4be None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:54.810476 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 169/337] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:02:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:02:54.815607 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9552a656-5693-4a61-bade-daa82162cda1 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:54.815873 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9552a656-5693-4a61-bade-daa82162cda1 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:54.816153 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9552a656-5693-4a61-bade-daa82162cda1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:54.853831 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-9552a656-5693-4a61-bade-daa82162cda1 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:54.855026 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-9552a656-5693-4a61-bade-daa82162cda1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:54.878899 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:54.938844 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c653aa7-9a03-430e-9976-6c8869af29b4 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:54.977614 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 170/338] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 16:02:54.978916 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652faf81-f483-4f05-bcc1-73b7dfec7478 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:54.979199 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652faf81-f483-4f05-bcc1-73b7dfec7478 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:54.979472 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652faf81-f483-4f05-bcc1-73b7dfec7478 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:54.980878 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 171/339] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:02:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:02:54.986363 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e6763-77b7-41d6-afb0-b10e88f8a361 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:54.986645 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e6763-77b7-41d6-afb0-b10e88f8a361 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:54.986922 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e6763-77b7-41d6-afb0-b10e88f8a361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:55.012814 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f566d17d-cae1-4d51-93a7-d4fe8d48f492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:55.013567 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f566d17d-cae1-4d51-93a7-d4fe8d48f492 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:55.013888 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f566d17d-cae1-4d51-93a7-d4fe8d48f492 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:55.014186 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f566d17d-cae1-4d51-93a7-d4fe8d48f492 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:55.022583 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-d45e6763-77b7-41d6-afb0-b10e88f8a361 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:55.023720 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d45e6763-77b7-41d6-afb0-b10e88f8a361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:55.045778 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f566d17d-cae1-4d51-93a7-d4fe8d48f492 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:02:55.050869 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-f566d17d-cae1-4d51-93a7-d4fe8d48f492 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 16:02:55.050869 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:55.050869 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:55.050869 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:55.050869 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:55.050869 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:55.050869 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:55.050869 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:55.050869 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:55.050869 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:55.050869 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:55.050869 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:55.050869 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:02:55.050869 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:02:55.050869 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:02:55.050869 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:02:55.051741 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:02:55.051741 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:02:55.051741 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:02:55.051741 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:02:55.051741 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:55.051741 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:55.051741 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:55.051741 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:55.051741 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:55.051741 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:55.051741 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:55.051741 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:55.051741 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:55.051741 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:55.051741 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:55.051741 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:55.052542 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:55.052542 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:55.052542 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:55.052542 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:55.052542 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:02:55.052542 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:02:55.052542 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:02:55.052542 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:02:55.052542 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:02:55.052542 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:02:55.052542 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 16:02:55.052542 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:02:55.052542 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 169/340] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 6 16:02:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:02:55.063112 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f566d17d-cae1-4d51-93a7-d4fe8d48f492 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:55.108929 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 172/341] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:02:55.163030 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:55.228723 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f566d17d-cae1-4d51-93a7-d4fe8d48f492 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:55.276224 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4431153-bc33-474a-957e-df79594411e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:55.276778 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4431153-bc33-474a-957e-df79594411e5 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:55.277523 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4431153-bc33-474a-957e-df79594411e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:55.277523 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4431153-bc33-474a-957e-df79594411e5 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:55.277970 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4431153-bc33-474a-957e-df79594411e5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:55.278337 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4431153-bc33-474a-957e-df79594411e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:55.279755 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4431153-bc33-474a-957e-df79594411e5 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:55.292388 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 170/342] 10.222.0.43 () {56 vars in 1101 bytes} [Tue Sep 6 16:02:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:02:55.298076 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45e6763-77b7-41d6-afb0-b10e88f8a361 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:55.394835 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:55.397014 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4431153-bc33-474a-957e-df79594411e5 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:55.424928 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45e6763-77b7-41d6-afb0-b10e88f8a361 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:55.467817 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc35945-82d1-4a93-b5eb-9430547040cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:55.468443 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc35945-82d1-4a93-b5eb-9430547040cb None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:55.468714 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc35945-82d1-4a93-b5eb-9430547040cb None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:55.469001 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc35945-82d1-4a93-b5eb-9430547040cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:55.552658 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc35945-82d1-4a93-b5eb-9430547040cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:55.554176 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc35945-82d1-4a93-b5eb-9430547040cb None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:55.594410 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4431153-bc33-474a-957e-df79594411e5 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:55.625721 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:55.651048 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:55.670518 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917cf3f8-51f0-4698-a1bd-3158120bd21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:55.671220 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917cf3f8-51f0-4698-a1bd-3158120bd21c None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:55.671496 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917cf3f8-51f0-4698-a1bd-3158120bd21c None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:55.671764 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917cf3f8-51f0-4698-a1bd-3158120bd21c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:55.695001 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917cf3f8-51f0-4698-a1bd-3158120bd21c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:02:55.696725 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 173/343] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 6 16:02:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 16:02:55.698190 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b84f2d-b190-4a9e-8a77-c1ea635fbd1a None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:55.698481 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b84f2d-b190-4a9e-8a77-c1ea635fbd1a None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:55.698750 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b84f2d-b190-4a9e-8a77-c1ea635fbd1a None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:55.700258 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 174/344] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:02:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:02:55.700984 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b84f2d-b190-4a9e-8a77-c1ea635fbd1a None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:55.704982 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-917cf3f8-51f0-4698-a1bd-3158120bd21c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 16:02:55.704982 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:55.704982 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:55.704982 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:55.704982 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:55.704982 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:55.704982 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:55.704982 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:55.704982 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:55.704982 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:55.704982 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:55.704982 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:55.704982 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:02:55.704982 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:02:55.704982 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:02:55.704982 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:02:55.706310 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:02:55.706310 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:02:55.706310 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:02:55.706310 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:02:55.706310 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:55.706310 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:55.706310 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:55.706310 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:55.706310 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:55.706310 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:55.706310 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:55.706310 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:55.706310 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:55.706310 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:55.706310 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:55.706310 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:55.707446 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:55.707446 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:55.707446 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:55.707446 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:55.707446 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:02:55.707446 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:02:55.707446 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:02:55.707446 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:02:55.707446 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:02:55.707446 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:02:55.707446 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 16:02:55.707446 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:02:55.707446 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 171/345] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 6 16:02:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:02:55.709689 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d50b23-162e-4aff-a147-d0afce128506 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:55.710941 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d50b23-162e-4aff-a147-d0afce128506 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:55.711226 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d50b23-162e-4aff-a147-d0afce128506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:55.733350 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-72d50b23-162e-4aff-a147-d0afce128506 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:55.737606 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-72d50b23-162e-4aff-a147-d0afce128506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:55.825733 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 172/346] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:02:55.826906 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d50b23-162e-4aff-a147-d0afce128506 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:55.831333 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b84f2d-b190-4a9e-8a77-c1ea635fbd1a None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:55.882381 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af7654b-7196-4bd5-be79-0f295eeded82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:55.883818 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af7654b-7196-4bd5-be79-0f295eeded82 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:55.884243 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af7654b-7196-4bd5-be79-0f295eeded82 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:55.884647 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af7654b-7196-4bd5-be79-0f295eeded82 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:55.904638 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af7654b-7196-4bd5-be79-0f295eeded82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:02:55.909917 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-5af7654b-7196-4bd5-be79-0f295eeded82 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 16:02:55.909917 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:55.909917 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:55.909917 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:55.909917 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:55.909917 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:55.909917 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:55.909917 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:55.909917 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:55.909917 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:55.909917 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:55.909917 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:55.909917 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:02:55.909917 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:02:55.909917 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:02:55.909917 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:02:55.911025 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:02:55.911025 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:02:55.911025 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:02:55.911025 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:02:55.911025 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:55.911025 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:55.911025 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:55.911025 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:55.911025 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:55.911025 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:55.911025 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:55.911025 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:55.911025 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:55.911025 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:55.911025 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:55.911025 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:55.912152 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:55.912152 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:55.912152 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:55.912152 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:55.912152 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:02:55.912152 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:02:55.912152 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:02:55.912152 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:02:55.912152 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:02:55.912152 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:02:55.912152 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 16:02:55.912152 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:02:55.918077 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 175/347] 10.222.0.43 () {56 vars in 1114 bytes} [Tue Sep 6 16:02:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:02:55.919156 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af7654b-7196-4bd5-be79-0f295eeded82 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:55.950766 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d50b23-162e-4aff-a147-d0afce128506 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:55.953220 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:55.976874 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:55.989110 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4d1e3d-2218-4f05-ad1c-70c9edaacf4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:55.989805 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4d1e3d-2218-4f05-ad1c-70c9edaacf4f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:55.994948 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4d1e3d-2218-4f05-ad1c-70c9edaacf4f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:55.994948 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4d1e3d-2218-4f05-ad1c-70c9edaacf4f None admin] PATH_INFO: `/v3/projects/5e7a87d573a8446bbcbf9e28149a392f` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:55.994948 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4d1e3d-2218-4f05-ad1c-70c9edaacf4f None admin] RBAC: Authorizing `identity:get_project(project_id=5e7a87d573a8446bbcbf9e28149a392f)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:55.994948 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4d1e3d-2218-4f05-ad1c-70c9edaacf4f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:55.994948 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 173/348] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 6 16:02:55 2022] GET /identity/v3/projects/5e7a87d573a8446bbcbf9e28149a392f => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:55.995691 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4d1e3d-2218-4f05-ad1c-70c9edaacf4f None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:56.057095 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af7654b-7196-4bd5-be79-0f295eeded82 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:56.110645 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693e5cbc-71bd-4443-ae0d-371698c7e9c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:56.111200 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693e5cbc-71bd-4443-ae0d-371698c7e9c9 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:56.111475 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693e5cbc-71bd-4443-ae0d-371698c7e9c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:56.111704 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693e5cbc-71bd-4443-ae0d-371698c7e9c9 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:56.112380 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693e5cbc-71bd-4443-ae0d-371698c7e9c9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:56.112675 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693e5cbc-71bd-4443-ae0d-371698c7e9c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:56.114102 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693e5cbc-71bd-4443-ae0d-371698c7e9c9 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:56.128807 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 176/349] 10.222.0.43 () {56 vars in 1101 bytes} [Tue Sep 6 16:02:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:56.129294 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693e5cbc-71bd-4443-ae0d-371698c7e9c9 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:56.129829 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4d1e3d-2218-4f05-ad1c-70c9edaacf4f None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:56.186352 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c205bb55-34b8-4bd5-911c-92fb764edf57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:56.187594 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c205bb55-34b8-4bd5-911c-92fb764edf57 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:56.188086 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c205bb55-34b8-4bd5-911c-92fb764edf57 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:56.188843 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c205bb55-34b8-4bd5-911c-92fb764edf57 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:56.190135 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c205bb55-34b8-4bd5-911c-92fb764edf57 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:56.190631 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c205bb55-34b8-4bd5-911c-92fb764edf57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:56.192804 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c205bb55-34b8-4bd5-911c-92fb764edf57 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:56.205344 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 174/350] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 6 16:02:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:56.207177 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c205bb55-34b8-4bd5-911c-92fb764edf57 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:56.272054 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:56.278849 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:56.285093 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693e5cbc-71bd-4443-ae0d-371698c7e9c9 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:56.326505 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a8bee1-a0c6-44d5-b742-848d06df2a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:56.327147 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a8bee1-a0c6-44d5-b742-848d06df2a17 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:56.327425 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a8bee1-a0c6-44d5-b742-848d06df2a17 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:56.327729 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a8bee1-a0c6-44d5-b742-848d06df2a17 None admin] PATH_INFO: `/v3/groups/b9a831cedb0a426580bacc6913f70b8f` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:56.338050 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a8bee1-a0c6-44d5-b742-848d06df2a17 None admin] RBAC: Authorizing `identity:get_group(group_id=b9a831cedb0a426580bacc6913f70b8f)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:56.339656 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a8bee1-a0c6-44d5-b742-848d06df2a17 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:56.341385 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 177/351] 10.222.0.43 () {56 vars in 1180 bytes} [Tue Sep 6 16:02:56 2022] GET /identity/v3/groups/b9a831cedb0a426580bacc6913f70b8f => generated 226 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:56.342313 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a8bee1-a0c6-44d5-b742-848d06df2a17 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:56.364739 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c205bb55-34b8-4bd5-911c-92fb764edf57 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:56.406757 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88025653-62ca-4428-9f4b-4dfa61c843f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:56.407515 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88025653-62ca-4428-9f4b-4dfa61c843f5 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:56.407817 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88025653-62ca-4428-9f4b-4dfa61c843f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:56.408326 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88025653-62ca-4428-9f4b-4dfa61c843f5 None admin] PATH_INFO: `/v3/users/f1ebbcd3c6be44178b34f78d12b202a2` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:56.409879 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88025653-62ca-4428-9f4b-4dfa61c843f5 None admin] RBAC: Authorizing `identity:get_user(user_id=f1ebbcd3c6be44178b34f78d12b202a2)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:56.411989 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88025653-62ca-4428-9f4b-4dfa61c843f5 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:56.414743 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 175/352] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 6 16:02:56 2022] GET /identity/v3/users/f1ebbcd3c6be44178b34f78d12b202a2 => generated 293 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:56.415293 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88025653-62ca-4428-9f4b-4dfa61c843f5 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:56.459367 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a8bee1-a0c6-44d5-b742-848d06df2a17 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:56.501243 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb7cede-ef14-4ba8-8e7d-534840dc993f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:56.502246 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb7cede-ef14-4ba8-8e7d-534840dc993f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:56.502756 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb7cede-ef14-4ba8-8e7d-534840dc993f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:56.503190 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb7cede-ef14-4ba8-8e7d-534840dc993f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:56.539063 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88025653-62ca-4428-9f4b-4dfa61c843f5 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:56.540606 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:56.544431 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:56.581289 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213f0523-5633-468b-98f3-d4e923798e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:56.582264 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f0523-5633-468b-98f3-d4e923798e7e None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:56.582403 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f0523-5633-468b-98f3-d4e923798e7e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:56.582685 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f0523-5633-468b-98f3-d4e923798e7e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:56.594890 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213f0523-5633-468b-98f3-d4e923798e7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:02:56.595774 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb7cede-ef14-4ba8-8e7d-534840dc993f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:56.597427 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb7cede-ef14-4ba8-8e7d-534840dc993f None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:56.601329 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-213f0523-5633-468b-98f3-d4e923798e7e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 16:02:56.601329 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:56.601329 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:56.601329 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:56.601329 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:56.601329 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:56.601329 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:56.601329 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:56.601329 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:56.601329 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:56.601329 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:56.601329 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:56.601329 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:02:56.601329 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:02:56.601329 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:02:56.601329 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:02:56.602395 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:02:56.602395 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:02:56.602395 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:02:56.602395 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:02:56.602395 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:56.602395 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:56.602395 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:56.602395 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:56.602395 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:56.602395 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:56.602395 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:56.602395 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:56.602395 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:56.602395 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:56.602395 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:56.602395 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:56.603300 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:56.603300 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:56.603300 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:56.603300 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:56.603300 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:02:56.603300 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:02:56.603300 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:02:56.603300 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:02:56.603300 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:02:56.603300 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:02:56.603300 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 16:02:56.603300 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:02:56.603300 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 176/353] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 6 16:02:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:02:56.608831 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213f0523-5633-468b-98f3-d4e923798e7e None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:56.721567 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 178/354] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 6 16:02:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 16:02:56.722170 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb7cede-ef14-4ba8-8e7d-534840dc993f None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:56.734302 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213f0523-5633-468b-98f3-d4e923798e7e None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:56.777643 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355027c0-8f30-48ee-8f58-8848cd0de77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:56.778259 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355027c0-8f30-48ee-8f58-8848cd0de77d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:56.778533 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355027c0-8f30-48ee-8f58-8848cd0de77d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:56.778800 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355027c0-8f30-48ee-8f58-8848cd0de77d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:56.779535 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355027c0-8f30-48ee-8f58-8848cd0de77d None admin] RBAC: Adding query filter params (group.id=b9a831cedb0a426580bacc6913f70b8f, role.id=15252b0b89894c1e8a6bd9a98b1c3ab7, scope.project.id=5e7a87d573a8446bbcbf9e28149a392f) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:56.779837 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355027c0-8f30-48ee-8f58-8848cd0de77d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:56.781435 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355027c0-8f30-48ee-8f58-8848cd0de77d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:56.787239 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 177/355] 10.222.0.43 () {56 vars in 1376 bytes} [Tue Sep 6 16:02:56 2022] GET /identity/v3/role_assignments?group.id=b9a831cedb0a426580bacc6913f70b8f&scope.project.id=5e7a87d573a8446bbcbf9e28149a392f&role.id=15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 260 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:56.788039 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355027c0-8f30-48ee-8f58-8848cd0de77d None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:56.812994 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:56.826451 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:56.874862 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb7cede-ef14-4ba8-8e7d-534840dc993f None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:56.923559 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5054884a-d115-4a7c-b650-1db925f22249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:56.924210 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5054884a-d115-4a7c-b650-1db925f22249 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:56.924771 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5054884a-d115-4a7c-b650-1db925f22249 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:56.925059 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5054884a-d115-4a7c-b650-1db925f22249 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:56.926317 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5054884a-d115-4a7c-b650-1db925f22249 None admin] RBAC: Adding query filter params (role.id=4908d6a02c4049e88249e9afbf7e2e9f, user.id=f1ebbcd3c6be44178b34f78d12b202a2, scope.system=all) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:56.926895 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5054884a-d115-4a7c-b650-1db925f22249 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:56.929165 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5054884a-d115-4a7c-b650-1db925f22249 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:56.931809 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355027c0-8f30-48ee-8f58-8848cd0de77d None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:56.945024 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 179/356] 10.222.0.43 () {56 vars in 1308 bytes} [Tue Sep 6 16:02:56 2022] GET /identity/v3/role_assignments?user.id=f1ebbcd3c6be44178b34f78d12b202a2&scope.system=all&role.id=4908d6a02c4049e88249e9afbf7e2e9f => generated 510 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:56.947817 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8ec631-608e-4d30-ac08-93e91a4f6f46 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:56.948327 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8ec631-608e-4d30-ac08-93e91a4f6f46 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:56.948849 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8ec631-608e-4d30-ac08-93e91a4f6f46 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:56.953328 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 180/357] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:02:56 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:02:56.955092 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8ec631-608e-4d30-ac08-93e91a4f6f46 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:56.988666 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23839802-1953-4a58-8e69-39e64487bc05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:56.989293 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23839802-1953-4a58-8e69-39e64487bc05 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:56.989564 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23839802-1953-4a58-8e69-39e64487bc05 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:56.989833 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23839802-1953-4a58-8e69-39e64487bc05 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:56.990560 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23839802-1953-4a58-8e69-39e64487bc05 None admin] RBAC: Adding query filter params (name=member) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:56.990857 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23839802-1953-4a58-8e69-39e64487bc05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:56.992261 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23839802-1953-4a58-8e69-39e64487bc05 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:57.012392 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 178/358] 10.222.0.43 () {56 vars in 1101 bytes} [Tue Sep 6 16:02:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:57.015082 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e89432-3da2-486c-90ce-67750d1343a6 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:57.015372 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e89432-3da2-486c-90ce-67750d1343a6 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:57.015632 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e89432-3da2-486c-90ce-67750d1343a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:57.055232 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-75e89432-3da2-486c-90ce-67750d1343a6 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:57.057822 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-75e89432-3da2-486c-90ce-67750d1343a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:57.153859 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8ec631-608e-4d30-ac08-93e91a4f6f46 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:57.181280 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 179/359] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:02:57.182157 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e89432-3da2-486c-90ce-67750d1343a6 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:57.182720 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:57.199634 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab7b2f1-5ca3-4160-bd3b-b78a938713c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:57.200611 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7b2f1-5ca3-4160-bd3b-b78a938713c2 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:57.201042 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7b2f1-5ca3-4160-bd3b-b78a938713c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:57.201443 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7b2f1-5ca3-4160-bd3b-b78a938713c2 None admin] PATH_INFO: `/v3/projects/5e7a87d573a8446bbcbf9e28149a392f` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:57.203181 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab7b2f1-5ca3-4160-bd3b-b78a938713c2 None admin] RBAC: Authorizing `identity:get_project(project_id=5e7a87d573a8446bbcbf9e28149a392f)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:57.205463 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab7b2f1-5ca3-4160-bd3b-b78a938713c2 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:57.208323 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 181/360] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 6 16:02:56 2022] GET /identity/v3/projects/5e7a87d573a8446bbcbf9e28149a392f => generated 298 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:57.214017 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:57.214800 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab7b2f1-5ca3-4160-bd3b-b78a938713c2 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:57.363100 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e89432-3da2-486c-90ce-67750d1343a6 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:57.398359 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab7b2f1-5ca3-4160-bd3b-b78a938713c2 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:57.411312 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30c3e4e-5709-49d6-9815-585fd25a0fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:57.412235 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30c3e4e-5709-49d6-9815-585fd25a0fcb None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:57.412513 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30c3e4e-5709-49d6-9815-585fd25a0fcb None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:57.412801 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30c3e4e-5709-49d6-9815-585fd25a0fcb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:57.439619 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be9709b-8488-474f-8160-04ff56fad102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:57.440253 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be9709b-8488-474f-8160-04ff56fad102 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:57.440529 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be9709b-8488-474f-8160-04ff56fad102 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:57.440894 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be9709b-8488-474f-8160-04ff56fad102 None admin] PATH_INFO: `/v3/groups/b9a831cedb0a426580bacc6913f70b8f` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:57.442441 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be9709b-8488-474f-8160-04ff56fad102 None admin] RBAC: Authorizing `identity:get_group(group_id=b9a831cedb0a426580bacc6913f70b8f)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:57.444467 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be9709b-8488-474f-8160-04ff56fad102 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:57.446715 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 182/361] 10.222.0.43 () {56 vars in 1180 bytes} [Tue Sep 6 16:02:57 2022] GET /identity/v3/groups/b9a831cedb0a426580bacc6913f70b8f => generated 226 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:02:57.447899 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb638233-59b1-43e6-a989-e411924d9968 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:57.448173 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb638233-59b1-43e6-a989-e411924d9968 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:57.448439 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb638233-59b1-43e6-a989-e411924d9968 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:57.449719 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 183/362] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:02:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:02:57.452858 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb638233-59b1-43e6-a989-e411924d9968 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:57.500487 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30c3e4e-5709-49d6-9815-585fd25a0fcb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:57.502036 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30c3e4e-5709-49d6-9815-585fd25a0fcb None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:57.516064 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:57.522115 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:57.569101 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb638233-59b1-43e6-a989-e411924d9968 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:57.612489 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f07018-c1bc-4e1e-b92e-6dbcde18bc1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:57.613125 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f07018-c1bc-4e1e-b92e-6dbcde18bc1b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:57.613397 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f07018-c1bc-4e1e-b92e-6dbcde18bc1b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:57.613666 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f07018-c1bc-4e1e-b92e-6dbcde18bc1b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:57.614380 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f07018-c1bc-4e1e-b92e-6dbcde18bc1b None admin] RBAC: Adding query filter params (group.id=b9a831cedb0a426580bacc6913f70b8f, role.id=61df19b017494bdcadc9a5e9d8bb9e52, scope.project.id=5e7a87d573a8446bbcbf9e28149a392f) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:57.614680 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f07018-c1bc-4e1e-b92e-6dbcde18bc1b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:57.616187 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f07018-c1bc-4e1e-b92e-6dbcde18bc1b None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:57.627238 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 184/363] 10.222.0.43 () {56 vars in 1376 bytes} [Tue Sep 6 16:02:57 2022] GET /identity/v3/role_assignments?group.id=b9a831cedb0a426580bacc6913f70b8f&scope.project.id=5e7a87d573a8446bbcbf9e28149a392f&role.id=61df19b017494bdcadc9a5e9d8bb9e52 => generated 260 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:02:57.628846 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01913bf5-eec6-4ea6-b513-deeddfe1e215 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:57.629062 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01913bf5-eec6-4ea6-b513-deeddfe1e215 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:57.630785 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01913bf5-eec6-4ea6-b513-deeddfe1e215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:57.630899 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 180/364] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 6 16:02:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 16:02:57.631424 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30c3e4e-5709-49d6-9815-585fd25a0fcb None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:57.653512 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-01913bf5-eec6-4ea6-b513-deeddfe1e215 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:57.655494 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-01913bf5-eec6-4ea6-b513-deeddfe1e215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:57.747540 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 185/365] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:02:57.749132 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01913bf5-eec6-4ea6-b513-deeddfe1e215 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:57.765907 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30c3e4e-5709-49d6-9815-585fd25a0fcb None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:57.804816 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:57.813472 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:57.842903 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afba71b-1ff7-43f1-8340-9115e3bcfb53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:57.843465 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afba71b-1ff7-43f1-8340-9115e3bcfb53 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:57.843737 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afba71b-1ff7-43f1-8340-9115e3bcfb53 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:57.843999 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afba71b-1ff7-43f1-8340-9115e3bcfb53 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:57.865546 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01913bf5-eec6-4ea6-b513-deeddfe1e215 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:57.865867 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afba71b-1ff7-43f1-8340-9115e3bcfb53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:02:57.872615 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-6afba71b-1ff7-43f1-8340-9115e3bcfb53 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 16:02:57.872615 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:57.872615 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:57.872615 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:57.872615 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:57.872615 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:57.872615 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:57.872615 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:57.872615 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:57.872615 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:57.872615 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:57.872615 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:57.872615 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:02:57.872615 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:02:57.872615 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:02:57.872615 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:02:57.873536 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:02:57.873536 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:02:57.873536 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:02:57.873536 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:02:57.873536 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:57.873536 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:57.873536 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:57.873536 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:57.873536 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:57.873536 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:57.873536 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:57.873536 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:57.873536 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:57.873536 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:57.873536 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:57.873536 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:57.874786 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:57.874786 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:57.874786 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:57.874786 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:57.874786 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:02:57.874786 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:02:57.874786 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:02:57.874786 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:02:57.874786 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:02:57.874786 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:02:57.874786 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 16:02:57.874786 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:02:57.874786 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 181/366] 10.222.0.43 () {56 vars in 1114 bytes} [Tue Sep 6 16:02:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:02:57.879860 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afba71b-1ff7-43f1-8340-9115e3bcfb53 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:57.902847 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c4af4f-fa01-43e4-a13e-9026a0e6d85f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:57.903492 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c4af4f-fa01-43e4-a13e-9026a0e6d85f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:57.903763 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c4af4f-fa01-43e4-a13e-9026a0e6d85f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:57.904027 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c4af4f-fa01-43e4-a13e-9026a0e6d85f None admin] PATH_INFO: `/v3/projects/cf9744296b5341f08017259b44d86f94` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:57.905188 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c4af4f-fa01-43e4-a13e-9026a0e6d85f None admin] RBAC: Authorizing `identity:get_project(project_id=cf9744296b5341f08017259b44d86f94)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:57.906929 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c4af4f-fa01-43e4-a13e-9026a0e6d85f None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:57.909184 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 186/367] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 6 16:02:57 2022] GET /identity/v3/projects/cf9744296b5341f08017259b44d86f94 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:57.916643 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c4af4f-fa01-43e4-a13e-9026a0e6d85f None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:58.012684 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afba71b-1ff7-43f1-8340-9115e3bcfb53 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:58.023785 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c4af4f-fa01-43e4-a13e-9026a0e6d85f None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:58.046061 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:58.051982 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958affe0-22ab-4268-943d-cf6b38300cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:58.053220 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958affe0-22ab-4268-943d-cf6b38300cb0 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:58.054247 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958affe0-22ab-4268-943d-cf6b38300cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:58.054677 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958affe0-22ab-4268-943d-cf6b38300cb0 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:58.055680 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958affe0-22ab-4268-943d-cf6b38300cb0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:58.056160 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958affe0-22ab-4268-943d-cf6b38300cb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:58.057886 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958affe0-22ab-4268-943d-cf6b38300cb0 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:58.062049 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4927376c-4c08-4baf-8ce2-e3d4b8682a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:58.062688 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927376c-4c08-4baf-8ce2-e3d4b8682a59 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:58.062961 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927376c-4c08-4baf-8ce2-e3d4b8682a59 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:58.063228 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927376c-4c08-4baf-8ce2-e3d4b8682a59 None admin] PATH_INFO: `/v3/groups/b9a831cedb0a426580bacc6913f70b8f` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:58.064718 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4927376c-4c08-4baf-8ce2-e3d4b8682a59 None admin] RBAC: Authorizing `identity:get_group(group_id=b9a831cedb0a426580bacc6913f70b8f)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:58.065914 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4927376c-4c08-4baf-8ce2-e3d4b8682a59 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:58.067790 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 187/368] 10.222.0.43 () {56 vars in 1180 bytes} [Tue Sep 6 16:02:57 2022] GET /identity/v3/groups/b9a831cedb0a426580bacc6913f70b8f => generated 226 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:58.068200 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 182/369] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 6 16:02:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:02:58.073862 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4927376c-4c08-4baf-8ce2-e3d4b8682a59 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:58.073992 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958affe0-22ab-4268-943d-cf6b38300cb0 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:58.119092 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:58.182784 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958affe0-22ab-4268-943d-cf6b38300cb0 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:58.185269 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4927376c-4c08-4baf-8ce2-e3d4b8682a59 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:58.218957 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d04eef-19b9-4e8a-a5f9-e16cb785fe23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:58.219567 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d04eef-19b9-4e8a-a5f9-e16cb785fe23 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:58.219859 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d04eef-19b9-4e8a-a5f9-e16cb785fe23 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:58.220105 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d04eef-19b9-4e8a-a5f9-e16cb785fe23 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:58.220876 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d04eef-19b9-4e8a-a5f9-e16cb785fe23 None admin] RBAC: Adding query filter params (group.id=b9a831cedb0a426580bacc6913f70b8f, role.id=15252b0b89894c1e8a6bd9a98b1c3ab7, scope.project.id=cf9744296b5341f08017259b44d86f94) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:58.221179 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d04eef-19b9-4e8a-a5f9-e16cb785fe23 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:58.222565 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfca4a1-99a5-4eba-abb6-b5099e6233a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:58.222682 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d04eef-19b9-4e8a-a5f9-e16cb785fe23 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:58.223482 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfca4a1-99a5-4eba-abb6-b5099e6233a8 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:58.223959 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfca4a1-99a5-4eba-abb6-b5099e6233a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:58.224576 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfca4a1-99a5-4eba-abb6-b5099e6233a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:58.228939 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 183/370] 10.222.0.43 () {56 vars in 1376 bytes} [Tue Sep 6 16:02:58 2022] GET /identity/v3/role_assignments?group.id=b9a831cedb0a426580bacc6913f70b8f&scope.project.id=cf9744296b5341f08017259b44d86f94&role.id=15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 260 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:58.290338 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:58.303662 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfca4a1-99a5-4eba-abb6-b5099e6233a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:58.305497 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfca4a1-99a5-4eba-abb6-b5099e6233a8 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:58.334912 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d04eef-19b9-4e8a-a5f9-e16cb785fe23 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:58.416708 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 188/371] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 6 16:02:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 16:02:58.422745 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfca4a1-99a5-4eba-abb6-b5099e6233a8 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:58.518889 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d04eef-19b9-4e8a-a5f9-e16cb785fe23 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:58.522863 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:58.544413 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:58.565450 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfca4a1-99a5-4eba-abb6-b5099e6233a8 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:58.597644 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab01b757-cdbc-49d0-8875-aa2c1e939f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:58.598292 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01b757-cdbc-49d0-8875-aa2c1e939f2d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:58.598562 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01b757-cdbc-49d0-8875-aa2c1e939f2d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:58.598825 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01b757-cdbc-49d0-8875-aa2c1e939f2d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:58.616434 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab01b757-cdbc-49d0-8875-aa2c1e939f2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:02:58.625670 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eed4dab-da48-4c95-842e-f1a9f26ab9aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:58.626305 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eed4dab-da48-4c95-842e-f1a9f26ab9aa None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:58.626577 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eed4dab-da48-4c95-842e-f1a9f26ab9aa None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:58.626844 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eed4dab-da48-4c95-842e-f1a9f26ab9aa None admin] PATH_INFO: `/v3/projects/cf9744296b5341f08017259b44d86f94` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:58.627809 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-ab01b757-cdbc-49d0-8875-aa2c1e939f2d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:02:58.627809 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:58.627809 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:58.627809 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:58.627809 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:58.627809 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:58.627809 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:58.627809 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:58.627809 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:58.627809 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:58.627809 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:58.627809 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:58.627809 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:02:58.627809 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:02:58.627809 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:02:58.627809 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:02:58.628669 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:02:58.628669 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:02:58.628669 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:02:58.628669 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:02:58.628669 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:58.628669 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:58.628669 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:58.628669 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:58.628669 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:58.628669 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:58.628669 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:58.628669 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:58.628669 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:58.628669 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:58.628669 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:58.628669 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:58.629521 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:58.629521 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:58.629521 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:58.629521 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:58.629521 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:02:58.629521 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:02:58.629521 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:02:58.629521 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:02:58.629521 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:02:58.629521 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:02:58.629521 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:02:58.629521 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:02:58.630121 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eed4dab-da48-4c95-842e-f1a9f26ab9aa None admin] RBAC: Authorizing `identity:get_project(project_id=cf9744296b5341f08017259b44d86f94)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:58.631143 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eed4dab-da48-4c95-842e-f1a9f26ab9aa None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:58.633103 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 184/372] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 6 16:02:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:02:58.633324 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 189/373] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 6 16:02:58 2022] GET /identity/v3/projects/cf9744296b5341f08017259b44d86f94 => generated 302 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:02:58.634803 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab01b757-cdbc-49d0-8875-aa2c1e939f2d None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:58.638829 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eed4dab-da48-4c95-842e-f1a9f26ab9aa None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:58.814285 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab01b757-cdbc-49d0-8875-aa2c1e939f2d None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:58.836928 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eed4dab-da48-4c95-842e-f1a9f26ab9aa None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:58.882798 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba7efd8-139a-4beb-a34b-2a4d63071667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:58.883418 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba7efd8-139a-4beb-a34b-2a4d63071667 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:58.883689 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba7efd8-139a-4beb-a34b-2a4d63071667 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:58.883956 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba7efd8-139a-4beb-a34b-2a4d63071667 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:58.884639 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba7efd8-139a-4beb-a34b-2a4d63071667 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:58.884963 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba7efd8-139a-4beb-a34b-2a4d63071667 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:58.886369 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba7efd8-139a-4beb-a34b-2a4d63071667 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:58.897007 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9154f4f-dfca-4b86-b25e-5abc16a24ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:58.897658 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9154f4f-dfca-4b86-b25e-5abc16a24ca1 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:58.897925 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9154f4f-dfca-4b86-b25e-5abc16a24ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:58.898189 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9154f4f-dfca-4b86-b25e-5abc16a24ca1 None admin] PATH_INFO: `/v3/groups/b9a831cedb0a426580bacc6913f70b8f` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:58.901675 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9154f4f-dfca-4b86-b25e-5abc16a24ca1 None admin] RBAC: Authorizing `identity:get_group(group_id=b9a831cedb0a426580bacc6913f70b8f)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:58.903129 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9154f4f-dfca-4b86-b25e-5abc16a24ca1 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:58.904925 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 190/374] 10.222.0.43 () {56 vars in 1180 bytes} [Tue Sep 6 16:02:58 2022] GET /identity/v3/groups/b9a831cedb0a426580bacc6913f70b8f => generated 226 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:58.907068 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 185/375] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 6 16:02:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:58.913836 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9154f4f-dfca-4b86-b25e-5abc16a24ca1 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:58.915175 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba7efd8-139a-4beb-a34b-2a4d63071667 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:58.962288 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:58.962486 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:59.053766 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba7efd8-139a-4beb-a34b-2a4d63071667 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:59.094364 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9154f4f-dfca-4b86-b25e-5abc16a24ca1 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:59.097759 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e5b55b-3bc8-40e6-8616-459bddbf67fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:59.098381 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e5b55b-3bc8-40e6-8616-459bddbf67fd None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:59.098658 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e5b55b-3bc8-40e6-8616-459bddbf67fd None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:59.098925 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e5b55b-3bc8-40e6-8616-459bddbf67fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:59.146401 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11e2896-d74c-45e8-9c4e-bf3a47a7e5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:59.147499 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e2896-d74c-45e8-9c4e-bf3a47a7e5f7 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:59.148840 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e2896-d74c-45e8-9c4e-bf3a47a7e5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:59.149258 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e2896-d74c-45e8-9c4e-bf3a47a7e5f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:59.150165 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11e2896-d74c-45e8-9c4e-bf3a47a7e5f7 None admin] RBAC: Adding query filter params (group.id=b9a831cedb0a426580bacc6913f70b8f, role.id=61df19b017494bdcadc9a5e9d8bb9e52, scope.project.id=cf9744296b5341f08017259b44d86f94) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:59.150612 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11e2896-d74c-45e8-9c4e-bf3a47a7e5f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:59.152193 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11e2896-d74c-45e8-9c4e-bf3a47a7e5f7 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:59.159653 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 191/376] 10.222.0.43 () {56 vars in 1376 bytes} [Tue Sep 6 16:02:58 2022] GET /identity/v3/role_assignments?group.id=b9a831cedb0a426580bacc6913f70b8f&scope.project.id=cf9744296b5341f08017259b44d86f94&role.id=61df19b017494bdcadc9a5e9d8bb9e52 => generated 260 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:59.161412 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0f94ff-ec7c-4aff-8bc8-da240816e445 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:59.162469 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0f94ff-ec7c-4aff-8bc8-da240816e445 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:59.162889 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0f94ff-ec7c-4aff-8bc8-da240816e445 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:59.165223 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 192/377] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:02:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:02:59.171523 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cacf24-dcf9-4596-8392-d10e2492e97c None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:59.172037 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cacf24-dcf9-4596-8392-d10e2492e97c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:59.172632 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cacf24-dcf9-4596-8392-d10e2492e97c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:59.189099 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e5b55b-3bc8-40e6-8616-459bddbf67fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:59.190673 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e5b55b-3bc8-40e6-8616-459bddbf67fd None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:59.198243 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-02cacf24-dcf9-4596-8392-d10e2492e97c None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:59.200297 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-02cacf24-dcf9-4596-8392-d10e2492e97c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:59.226367 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:59.308367 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 193/378] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:02:59.332909 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 186/379] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 6 16:02:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 16:02:59.335496 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cacf24-dcf9-4596-8392-d10e2492e97c None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:59.357133 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:59.460426 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cacf24-dcf9-4596-8392-d10e2492e97c None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:59.508641 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df760e19-a8ae-491e-9d78-6778d5d1a45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:59.509312 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df760e19-a8ae-491e-9d78-6778d5d1a45c None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:59.509584 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df760e19-a8ae-491e-9d78-6778d5d1a45c None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:59.509849 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df760e19-a8ae-491e-9d78-6778d5d1a45c None admin] PATH_INFO: `/v3/projects/0ad262a5a8a84066b8eba54c7ea32d39` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:59.513434 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df760e19-a8ae-491e-9d78-6778d5d1a45c None admin] RBAC: Authorizing `identity:get_project(project_id=0ad262a5a8a84066b8eba54c7ea32d39)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:59.515027 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df760e19-a8ae-491e-9d78-6778d5d1a45c None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:59.516995 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 194/380] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 6 16:02:59 2022] GET /identity/v3/projects/0ad262a5a8a84066b8eba54c7ea32d39 => generated 344 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:59.522584 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e5b55b-3bc8-40e6-8616-459bddbf67fd None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:59.668970 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e5b55b-3bc8-40e6-8616-459bddbf67fd None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:59.686514 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df760e19-a8ae-491e-9d78-6778d5d1a45c None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:59.707186 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:59.722080 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6936d83-58a1-4857-ae7d-c4e179e65071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:59.722733 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6936d83-58a1-4857-ae7d-c4e179e65071 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:59.723013 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6936d83-58a1-4857-ae7d-c4e179e65071 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:59.723288 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6936d83-58a1-4857-ae7d-c4e179e65071 None admin] PATH_INFO: `/v3/groups/500b3b5ebf3d4b368d6e48864f4b961d` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:59.733830 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6936d83-58a1-4857-ae7d-c4e179e65071 None admin] RBAC: Authorizing `identity:get_group(group_id=500b3b5ebf3d4b368d6e48864f4b961d)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:59.736137 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6936d83-58a1-4857-ae7d-c4e179e65071 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:59.739040 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 187/381] 10.222.0.43 () {56 vars in 1180 bytes} [Tue Sep 6 16:02:59 2022] GET /identity/v3/groups/500b3b5ebf3d4b368d6e48864f4b961d => generated 229 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:59.743732 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6936d83-58a1-4857-ae7d-c4e179e65071 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:59.857925 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:59.879969 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df760e19-a8ae-491e-9d78-6778d5d1a45c None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:59.926006 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775e3567-ced4-4048-8f1b-f0765f2a3004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:59.926930 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775e3567-ced4-4048-8f1b-f0765f2a3004 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:59.927215 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775e3567-ced4-4048-8f1b-f0765f2a3004 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:59.927527 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775e3567-ced4-4048-8f1b-f0765f2a3004 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:59.940990 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775e3567-ced4-4048-8f1b-f0765f2a3004 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:02:59.947414 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-775e3567-ced4-4048-8f1b-f0765f2a3004 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 16:02:59.947414 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:59.947414 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:59.947414 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:59.947414 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:59.947414 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:59.947414 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:59.947414 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:59.947414 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:59.947414 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:59.947414 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:59.947414 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:59.947414 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:02:59.947414 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:02:59.947414 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:02:59.947414 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:02:59.948292 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:02:59.948292 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:02:59.948292 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:02:59.948292 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:02:59.948292 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:59.948292 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:59.948292 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:59.948292 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:59.948292 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:59.948292 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:59.948292 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:59.948292 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:59.948292 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:59.948292 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:59.948292 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:59.948292 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:59.949133 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:59.949133 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:59.949133 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:59.949133 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:59.949133 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:02:59.949133 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:02:59.949133 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:02:59.949133 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:02:59.949133 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:02:59.949133 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:02:59.949133 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 16:02:59.949133 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:02:59.949133 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 195/382] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 6 16:02:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:02:59.959308 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775e3567-ced4-4048-8f1b-f0765f2a3004 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:59.960451 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6936d83-58a1-4857-ae7d-c4e179e65071 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:00.017491 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e246367d-9e6a-4599-b674-a813aa6ca8b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:00.019155 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e246367d-9e6a-4599-b674-a813aa6ca8b3 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:00.019815 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e246367d-9e6a-4599-b674-a813aa6ca8b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:00.021751 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e246367d-9e6a-4599-b674-a813aa6ca8b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:00.022699 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e246367d-9e6a-4599-b674-a813aa6ca8b3 None admin] RBAC: Adding query filter params (group.id=500b3b5ebf3d4b368d6e48864f4b961d, role.id=edaabdea8be84a7980fb2c651fe52f63, scope.project.id=0ad262a5a8a84066b8eba54c7ea32d39) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:00.023676 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e246367d-9e6a-4599-b674-a813aa6ca8b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:00.025532 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e246367d-9e6a-4599-b674-a813aa6ca8b3 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:00.038002 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 188/383] 10.222.0.43 () {56 vars in 1376 bytes} [Tue Sep 6 16:02:59 2022] GET /identity/v3/role_assignments?group.id=500b3b5ebf3d4b368d6e48864f4b961d&scope.project.id=0ad262a5a8a84066b8eba54c7ea32d39&role.id=edaabdea8be84a7980fb2c651fe52f63 => generated 260 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:00.104718 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775e3567-ced4-4048-8f1b-f0765f2a3004 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:00.142104 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93efa5f-cec8-4c7b-9921-43f12af12151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:00.143100 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93efa5f-cec8-4c7b-9921-43f12af12151 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:00.143585 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93efa5f-cec8-4c7b-9921-43f12af12151 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:00.144041 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93efa5f-cec8-4c7b-9921-43f12af12151 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:00.144958 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93efa5f-cec8-4c7b-9921-43f12af12151 None admin] RBAC: Adding query filter params (name=member) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:00.145473 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93efa5f-cec8-4c7b-9921-43f12af12151 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:00.147200 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93efa5f-cec8-4c7b-9921-43f12af12151 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:00.157079 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 196/384] 10.222.0.43 () {56 vars in 1101 bytes} [Tue Sep 6 16:02:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:00.163254 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e246367d-9e6a-4599-b674-a813aa6ca8b3 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:00.230173 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:00.257323 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4862375f-0536-4a93-8659-64b5dbf91dd3 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:00.257661 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4862375f-0536-4a93-8659-64b5dbf91dd3 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:00.257865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4862375f-0536-4a93-8659-64b5dbf91dd3 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:00.259932 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 197/385] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:03:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:03:00.265676 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94025166-3d69-4d6d-a288-23582958d7c1 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:00.266073 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94025166-3d69-4d6d-a288-23582958d7c1 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:00.266444 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94025166-3d69-4d6d-a288-23582958d7c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:00.289307 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e246367d-9e6a-4599-b674-a813aa6ca8b3 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:00.297075 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-94025166-3d69-4d6d-a288-23582958d7c1 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:03:00.298339 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-94025166-3d69-4d6d-a288-23582958d7c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:03:00.301928 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:00.352378 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3028037-15d0-41a0-b00b-c81b5a0b6e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:00.353306 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3028037-15d0-41a0-b00b-c81b5a0b6e68 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:00.353725 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3028037-15d0-41a0-b00b-c81b5a0b6e68 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:00.354187 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3028037-15d0-41a0-b00b-c81b5a0b6e68 None admin] PATH_INFO: `/v3/groups/b9a831cedb0a426580bacc6913f70b8f` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:00.355667 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3028037-15d0-41a0-b00b-c81b5a0b6e68 None admin] RBAC: Authorizing `identity:get_group(group_id=b9a831cedb0a426580bacc6913f70b8f)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:00.357344 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3028037-15d0-41a0-b00b-c81b5a0b6e68 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:00.359394 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 189/386] 10.222.0.43 () {56 vars in 1180 bytes} [Tue Sep 6 16:03:00 2022] GET /identity/v3/groups/b9a831cedb0a426580bacc6913f70b8f => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:00.365441 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3028037-15d0-41a0-b00b-c81b5a0b6e68 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:00.396922 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 198/387] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:03:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 16:03:00.496637 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3028037-15d0-41a0-b00b-c81b5a0b6e68 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:00.523167 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:00.536236 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a391bfeb-cbe6-4086-b2f2-d3842ab9dcaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:00.536965 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a391bfeb-cbe6-4086-b2f2-d3842ab9dcaf None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:00.537245 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a391bfeb-cbe6-4086-b2f2-d3842ab9dcaf None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:00.537516 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a391bfeb-cbe6-4086-b2f2-d3842ab9dcaf None admin] PATH_INFO: `/v3/projects/5e7a87d573a8446bbcbf9e28149a392f` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:00.538649 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a391bfeb-cbe6-4086-b2f2-d3842ab9dcaf None admin] RBAC: Authorizing `identity:get_project(project_id=5e7a87d573a8446bbcbf9e28149a392f)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:00.540107 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a391bfeb-cbe6-4086-b2f2-d3842ab9dcaf None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:00.542846 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 190/388] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 6 16:03:00 2022] GET /identity/v3/projects/5e7a87d573a8446bbcbf9e28149a392f => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:00.547811 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94025166-3d69-4d6d-a288-23582958d7c1 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:00.670344 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94025166-3d69-4d6d-a288-23582958d7c1 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:00.706301 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaf6a7b-567e-45a9-bdfc-bf51fbfacafd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:00.707043 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf6a7b-567e-45a9-bdfc-bf51fbfacafd None admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:00.707244 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf6a7b-567e-45a9-bdfc-bf51fbfacafd None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:00.707503 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf6a7b-567e-45a9-bdfc-bf51fbfacafd None admin] PATH_INFO: `/v3/projects/5e7a87d573a8446bbcbf9e28149a392f/groups/b9a831cedb0a426580bacc6913f70b8f/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:00.709460 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaf6a7b-567e-45a9-bdfc-bf51fbfacafd None admin] RBAC: Authorizing `identity:create_grant(project_id=5e7a87d573a8446bbcbf9e28149a392f, group_id=b9a831cedb0a426580bacc6913f70b8f, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:00.710907 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaf6a7b-567e-45a9-bdfc-bf51fbfacafd None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:00.729196 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:00.734789 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 199/389] 10.222.0.43 () {58 vars in 1442 bytes} [Tue Sep 6 16:03:00 2022] PUT /identity/v3/projects/5e7a87d573a8446bbcbf9e28149a392f/groups/b9a831cedb0a426580bacc6913f70b8f/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:03:00.778797 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74489fac-ca91-4a0d-8b7e-c4fea6093f12 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:00.779137 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74489fac-ca91-4a0d-8b7e-c4fea6093f12 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:00.779342 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74489fac-ca91-4a0d-8b7e-c4fea6093f12 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:00.780931 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 191/390] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:03:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 16:03:00.788024 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8d1ee4-1d24-4130-8cd5-4200f03d53ca None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:00.788316 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8d1ee4-1d24-4130-8cd5-4200f03d53ca None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:00.788568 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8d1ee4-1d24-4130-8cd5-4200f03d53ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:00.813777 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-3b8d1ee4-1d24-4130-8cd5-4200f03d53ca None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:03:00.815022 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-3b8d1ee4-1d24-4130-8cd5-4200f03d53ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:03:00.905431 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 200/391] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:03:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:03:00.946925 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74489fac-ca91-4a0d-8b7e-c4fea6093f12 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:01.058582 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74489fac-ca91-4a0d-8b7e-c4fea6093f12 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:01.093255 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9395c3d9-ec98-478c-a617-c4df2085ebde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:01.093940 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9395c3d9-ec98-478c-a617-c4df2085ebde None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:01.094220 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9395c3d9-ec98-478c-a617-c4df2085ebde None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:01.094487 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9395c3d9-ec98-478c-a617-c4df2085ebde None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:01.106216 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9395c3d9-ec98-478c-a617-c4df2085ebde None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:03:01.111094 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-9395c3d9-ec98-478c-a617-c4df2085ebde None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 16:03:01.111094 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:01.111094 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:01.111094 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:01.111094 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:01.111094 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:01.111094 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:01.111094 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:01.111094 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:01.111094 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:01.111094 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:01.111094 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:01.111094 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:03:01.111094 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:03:01.111094 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:03:01.111094 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:03:01.111976 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:03:01.111976 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:03:01.111976 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:03:01.111976 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:03:01.111976 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:01.111976 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:01.111976 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:01.111976 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:01.111976 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:01.111976 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:01.111976 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:01.111976 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:01.111976 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:01.111976 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:01.111976 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:01.111976 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:01.112837 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:01.112837 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:01.112837 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:01.112837 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:01.112837 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:03:01.112837 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:03:01.112837 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:03:01.112837 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:03:01.112837 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:03:01.112837 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:03:01.112837 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 16:03:01.112837 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:01.112837 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 192/392] 10.222.0.43 () {56 vars in 1114 bytes} [Tue Sep 6 16:03:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:01.120273 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8d1ee4-1d24-4130-8cd5-4200f03d53ca None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:01.234796 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8d1ee4-1d24-4130-8cd5-4200f03d53ca None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:01.262356 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:01.270589 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dd7bf8-f837-49cd-97e4-35e18a948591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:01.271257 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dd7bf8-f837-49cd-97e4-35e18a948591 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:01.274501 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dd7bf8-f837-49cd-97e4-35e18a948591 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:01.274501 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dd7bf8-f837-49cd-97e4-35e18a948591 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:01.274501 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dd7bf8-f837-49cd-97e4-35e18a948591 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:01.274501 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dd7bf8-f837-49cd-97e4-35e18a948591 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:01.275137 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dd7bf8-f837-49cd-97e4-35e18a948591 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:01.288037 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 201/393] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 6 16:03:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:01.293667 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9395c3d9-ec98-478c-a617-c4df2085ebde None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:01.333166 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dd7bf8-f837-49cd-97e4-35e18a948591 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:01.379979 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:01.475194 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dd7bf8-f837-49cd-97e4-35e18a948591 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:01.484743 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9395c3d9-ec98-478c-a617-c4df2085ebde None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:01.512645 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa705226-18b5-43d0-9a29-5f47604a7114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:01.513474 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa705226-18b5-43d0-9a29-5f47604a7114 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:01.514108 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa705226-18b5-43d0-9a29-5f47604a7114 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:01.514481 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa705226-18b5-43d0-9a29-5f47604a7114 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:01.526525 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa705226-18b5-43d0-9a29-5f47604a7114 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:03:01.534076 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-aa705226-18b5-43d0-9a29-5f47604a7114 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 16:03:01.534076 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:01.534076 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:01.534076 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:01.534076 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:01.534076 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:01.534076 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:01.534076 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:01.534076 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:01.534076 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:01.534076 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:01.534076 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:01.534076 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:03:01.534076 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:03:01.534076 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:03:01.534076 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:03:01.534990 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:03:01.534990 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:03:01.534990 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:03:01.534990 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:03:01.534990 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:01.534990 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:01.534990 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:01.534990 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:01.534990 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:01.534990 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:01.534990 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:01.534990 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:01.534990 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:01.534990 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:01.534990 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:01.534990 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:01.535787 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:01.535787 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:01.535787 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:01.535787 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:01.535787 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:03:01.535787 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:03:01.535787 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:03:01.535787 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:03:01.535787 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:03:01.535787 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:03:01.535787 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 16:03:01.535787 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:01.536375 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 202/394] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 6 16:03:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:03:01.541961 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa705226-18b5-43d0-9a29-5f47604a7114 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:01.550918 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d97dd9c-00c9-4cff-85d0-f1f9715f6fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:01.551545 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d97dd9c-00c9-4cff-85d0-f1f9715f6fa8 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:01.551818 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d97dd9c-00c9-4cff-85d0-f1f9715f6fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:01.552086 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d97dd9c-00c9-4cff-85d0-f1f9715f6fa8 None admin] PATH_INFO: `/v3/groups/b9a831cedb0a426580bacc6913f70b8f` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:01.553262 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d97dd9c-00c9-4cff-85d0-f1f9715f6fa8 None admin] RBAC: Authorizing `identity:get_group(group_id=b9a831cedb0a426580bacc6913f70b8f)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:01.554767 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d97dd9c-00c9-4cff-85d0-f1f9715f6fa8 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:01.556659 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 193/395] 10.222.0.43 () {56 vars in 1180 bytes} [Tue Sep 6 16:03:01 2022] GET /identity/v3/groups/b9a831cedb0a426580bacc6913f70b8f => generated 226 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:01.562780 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d97dd9c-00c9-4cff-85d0-f1f9715f6fa8 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:01.625291 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:01.680792 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa705226-18b5-43d0-9a29-5f47604a7114 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:01.700816 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d97dd9c-00c9-4cff-85d0-f1f9715f6fa8 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:01.721022 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868faf98-f357-4f6c-b876-c34e92b7dfcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:01.721622 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868faf98-f357-4f6c-b876-c34e92b7dfcb None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:01.722062 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868faf98-f357-4f6c-b876-c34e92b7dfcb None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:01.722344 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868faf98-f357-4f6c-b876-c34e92b7dfcb None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:01.722819 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:01.723014 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868faf98-f357-4f6c-b876-c34e92b7dfcb None admin] RBAC: Adding query filter params (name=member) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:01.723614 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868faf98-f357-4f6c-b876-c34e92b7dfcb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:01.725219 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868faf98-f357-4f6c-b876-c34e92b7dfcb None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:01.737659 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 203/396] 10.222.0.43 () {56 vars in 1101 bytes} [Tue Sep 6 16:03:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:01.739082 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8bf1b0-d4de-4cdb-8b35-a6fc1b24f0fc None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:01.739373 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8bf1b0-d4de-4cdb-8b35-a6fc1b24f0fc None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:01.739674 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8bf1b0-d4de-4cdb-8b35-a6fc1b24f0fc None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:01.741210 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 204/397] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:03:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:03:01.746530 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8bf1b0-d4de-4cdb-8b35-a6fc1b24f0fc None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:01.751570 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ddd509-42b1-4482-a29a-283ca57fbe1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:01.752218 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ddd509-42b1-4482-a29a-283ca57fbe1f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:01.752489 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ddd509-42b1-4482-a29a-283ca57fbe1f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:01.752780 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ddd509-42b1-4482-a29a-283ca57fbe1f None admin] PATH_INFO: `/v3/projects/5e7a87d573a8446bbcbf9e28149a392f` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:01.753908 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ddd509-42b1-4482-a29a-283ca57fbe1f None admin] RBAC: Authorizing `identity:get_project(project_id=5e7a87d573a8446bbcbf9e28149a392f)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:01.755431 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ddd509-42b1-4482-a29a-283ca57fbe1f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:01.757799 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 194/398] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 6 16:03:01 2022] GET /identity/v3/projects/5e7a87d573a8446bbcbf9e28149a392f => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:01.759312 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0277923d-fa78-4e6c-aa4f-1637af53f1b5 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:01.759591 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0277923d-fa78-4e6c-aa4f-1637af53f1b5 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:01.760262 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0277923d-fa78-4e6c-aa4f-1637af53f1b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:01.788299 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-0277923d-fa78-4e6c-aa4f-1637af53f1b5 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:03:01.789552 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0277923d-fa78-4e6c-aa4f-1637af53f1b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:03:01.867368 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 195/399] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:03:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:03:01.868233 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0277923d-fa78-4e6c-aa4f-1637af53f1b5 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:01.881806 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8bf1b0-d4de-4cdb-8b35-a6fc1b24f0fc None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:01.904797 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:01.934798 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb3dffb-6cb6-4e2b-bab8-786dc4c67819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:01.935868 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb3dffb-6cb6-4e2b-bab8-786dc4c67819 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:01.936369 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb3dffb-6cb6-4e2b-bab8-786dc4c67819 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:01.936856 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb3dffb-6cb6-4e2b-bab8-786dc4c67819 None admin] PATH_INFO: `/v3/groups/b9a831cedb0a426580bacc6913f70b8f` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:01.939513 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb3dffb-6cb6-4e2b-bab8-786dc4c67819 None admin] RBAC: Authorizing `identity:get_group(group_id=b9a831cedb0a426580bacc6913f70b8f)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:01.941202 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb3dffb-6cb6-4e2b-bab8-786dc4c67819 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:01.943371 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 205/400] 10.222.0.43 () {56 vars in 1180 bytes} [Tue Sep 6 16:03:01 2022] GET /identity/v3/groups/b9a831cedb0a426580bacc6913f70b8f => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:01.964501 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb3dffb-6cb6-4e2b-bab8-786dc4c67819 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:01.979463 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0277923d-fa78-4e6c-aa4f-1637af53f1b5 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:01.998418 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:02.019189 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b3265d-205e-40a6-a9ff-348501652980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:02.019860 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b3265d-205e-40a6-a9ff-348501652980 None admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:02.020409 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b3265d-205e-40a6-a9ff-348501652980 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:02.020619 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b3265d-205e-40a6-a9ff-348501652980 None admin] PATH_INFO: `/v3/projects/5e7a87d573a8446bbcbf9e28149a392f/groups/b9a831cedb0a426580bacc6913f70b8f/roles/61df19b017494bdcadc9a5e9d8bb9e52` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:02.023002 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b3265d-205e-40a6-a9ff-348501652980 None admin] RBAC: Authorizing `identity:create_grant(project_id=5e7a87d573a8446bbcbf9e28149a392f, group_id=b9a831cedb0a426580bacc6913f70b8f, role_id=61df19b017494bdcadc9a5e9d8bb9e52)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:02.024954 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b3265d-205e-40a6-a9ff-348501652980 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:02.044861 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 196/401] 10.222.0.43 () {58 vars in 1442 bytes} [Tue Sep 6 16:03:01 2022] PUT /identity/v3/projects/5e7a87d573a8446bbcbf9e28149a392f/groups/b9a831cedb0a426580bacc6913f70b8f/roles/61df19b017494bdcadc9a5e9d8bb9e52 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:03:02.083890 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb3dffb-6cb6-4e2b-bab8-786dc4c67819 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:02.121635 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6792c4b9-094c-4154-b453-e7be927f9a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:02.122654 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6792c4b9-094c-4154-b453-e7be927f9a10 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:02.123127 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6792c4b9-094c-4154-b453-e7be927f9a10 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:02.123589 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6792c4b9-094c-4154-b453-e7be927f9a10 None admin] PATH_INFO: `/v3/projects/cf9744296b5341f08017259b44d86f94` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:02.125050 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6792c4b9-094c-4154-b453-e7be927f9a10 None admin] RBAC: Authorizing `identity:get_project(project_id=cf9744296b5341f08017259b44d86f94)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:02.126840 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6792c4b9-094c-4154-b453-e7be927f9a10 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:02.128865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 206/402] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 6 16:03:01 2022] GET /identity/v3/projects/cf9744296b5341f08017259b44d86f94 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:03:02.135722 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b3265d-205e-40a6-a9ff-348501652980 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:02.255948 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b3265d-205e-40a6-a9ff-348501652980 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:02.299240 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33770f3c-cf8d-45d9-ad4b-9ba5075f0d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:02.299925 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33770f3c-cf8d-45d9-ad4b-9ba5075f0d01 None admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:02.300201 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33770f3c-cf8d-45d9-ad4b-9ba5075f0d01 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:02.300539 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33770f3c-cf8d-45d9-ad4b-9ba5075f0d01 None admin] PATH_INFO: `/v3/projects/cf9744296b5341f08017259b44d86f94/groups/b9a831cedb0a426580bacc6913f70b8f/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:02.302603 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33770f3c-cf8d-45d9-ad4b-9ba5075f0d01 None admin] RBAC: Authorizing `identity:create_grant(project_id=cf9744296b5341f08017259b44d86f94, group_id=b9a831cedb0a426580bacc6913f70b8f, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:02.304160 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33770f3c-cf8d-45d9-ad4b-9ba5075f0d01 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:02.312171 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6792c4b9-094c-4154-b453-e7be927f9a10 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:02.327402 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 197/403] 10.222.0.43 () {58 vars in 1442 bytes} [Tue Sep 6 16:03:02 2022] PUT /identity/v3/projects/cf9744296b5341f08017259b44d86f94/groups/b9a831cedb0a426580bacc6913f70b8f/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:03:02.381244 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:02.439321 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6792c4b9-094c-4154-b453-e7be927f9a10 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:02.476844 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76a1564-23dd-4522-a60d-98a4294f0be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:02.477634 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76a1564-23dd-4522-a60d-98a4294f0be2 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:02.477914 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76a1564-23dd-4522-a60d-98a4294f0be2 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:02.478189 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76a1564-23dd-4522-a60d-98a4294f0be2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:02.490651 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76a1564-23dd-4522-a60d-98a4294f0be2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:03:02.496362 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-a76a1564-23dd-4522-a60d-98a4294f0be2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 16:03:02.496362 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:02.496362 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:02.496362 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:02.496362 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:02.496362 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:02.496362 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:02.496362 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:02.496362 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:02.496362 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:02.496362 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:02.496362 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:02.496362 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:03:02.496362 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:03:02.496362 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:03:02.496362 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:03:02.497318 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:03:02.497318 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:03:02.497318 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:03:02.497318 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:03:02.497318 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:02.497318 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:02.497318 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:02.497318 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:02.497318 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:02.497318 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:02.497318 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:02.497318 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:02.497318 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:02.497318 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:02.497318 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:02.497318 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:02.498120 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:02.498120 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:02.498120 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:02.498120 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:02.498120 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:03:02.498120 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:03:02.498120 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:03:02.498120 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:03:02.498120 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:03:02.498120 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:03:02.498120 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 16:03:02.498120 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:02.498120 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 207/404] 10.222.0.43 () {56 vars in 1114 bytes} [Tue Sep 6 16:03:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:02.503818 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33770f3c-cf8d-45d9-ad4b-9ba5075f0d01 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:02.545382 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:02.608328 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33770f3c-cf8d-45d9-ad4b-9ba5075f0d01 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:02.645085 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165a4163-9331-4c01-9757-1a2085d5337b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:02.645714 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165a4163-9331-4c01-9757-1a2085d5337b None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:02.646054 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165a4163-9331-4c01-9757-1a2085d5337b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:02.646331 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165a4163-9331-4c01-9757-1a2085d5337b None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:02.647018 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165a4163-9331-4c01-9757-1a2085d5337b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:02.647318 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165a4163-9331-4c01-9757-1a2085d5337b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:02.648717 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165a4163-9331-4c01-9757-1a2085d5337b None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:02.657755 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 198/405] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 6 16:03:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:02.663477 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76a1564-23dd-4522-a60d-98a4294f0be2 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:02.766117 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb03211-6584-4c1a-8494-a28d0afb6188 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:02.766432 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb03211-6584-4c1a-8494-a28d0afb6188 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:02.766678 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb03211-6584-4c1a-8494-a28d0afb6188 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:02.768214 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 199/406] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:03:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:03:02.772647 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76a1564-23dd-4522-a60d-98a4294f0be2 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:02.773339 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aae9d1-b077-4954-afec-5f41508d01e1 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:02.773615 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aae9d1-b077-4954-afec-5f41508d01e1 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:02.773890 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aae9d1-b077-4954-afec-5f41508d01e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:02.785192 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:02.810118 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e326cd76-ceec-41a1-ae91-5e2105f2a2ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:02.810770 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e326cd76-ceec-41a1-ae91-5e2105f2a2ba None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:02.811057 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e326cd76-ceec-41a1-ae91-5e2105f2a2ba None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:02.811332 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e326cd76-ceec-41a1-ae91-5e2105f2a2ba None admin] PATH_INFO: `/v3/groups/b9a831cedb0a426580bacc6913f70b8f` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:02.812469 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e326cd76-ceec-41a1-ae91-5e2105f2a2ba None admin] RBAC: Authorizing `identity:get_group(group_id=b9a831cedb0a426580bacc6913f70b8f)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:02.814028 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e326cd76-ceec-41a1-ae91-5e2105f2a2ba None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:02.816055 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 208/407] 10.222.0.43 () {56 vars in 1180 bytes} [Tue Sep 6 16:03:02 2022] GET /identity/v3/groups/b9a831cedb0a426580bacc6913f70b8f => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:02.817206 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-23aae9d1-b077-4954-afec-5f41508d01e1 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:03:02.818481 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-23aae9d1-b077-4954-afec-5f41508d01e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:03:02.822557 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e326cd76-ceec-41a1-ae91-5e2105f2a2ba None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:02.928173 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e326cd76-ceec-41a1-ae91-5e2105f2a2ba None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:02.957970 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 200/408] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:03:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:03:02.962580 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e217a4-4f43-4e3c-a60f-a661af55679b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:02.963226 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e217a4-4f43-4e3c-a60f-a661af55679b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:02.963501 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e217a4-4f43-4e3c-a60f-a661af55679b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:02.963772 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e217a4-4f43-4e3c-a60f-a661af55679b None admin] PATH_INFO: `/v3/projects/cf9744296b5341f08017259b44d86f94` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:02.964970 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e217a4-4f43-4e3c-a60f-a661af55679b None admin] RBAC: Authorizing `identity:get_project(project_id=cf9744296b5341f08017259b44d86f94)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:02.966417 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e217a4-4f43-4e3c-a60f-a661af55679b None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:02.968340 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 209/409] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 6 16:03:02 2022] GET /identity/v3/projects/cf9744296b5341f08017259b44d86f94 => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:03:02.974114 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aae9d1-b077-4954-afec-5f41508d01e1 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:03.012358 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:03.107740 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aae9d1-b077-4954-afec-5f41508d01e1 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:03.149249 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f2ba14-24ed-4197-b69a-cebbe38eea4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:03.150221 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f2ba14-24ed-4197-b69a-cebbe38eea4b None admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:03.150687 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f2ba14-24ed-4197-b69a-cebbe38eea4b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:03.151133 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f2ba14-24ed-4197-b69a-cebbe38eea4b None admin] PATH_INFO: `/v3/projects/cf9744296b5341f08017259b44d86f94/groups/b9a831cedb0a426580bacc6913f70b8f/roles/61df19b017494bdcadc9a5e9d8bb9e52` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:03.153562 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f2ba14-24ed-4197-b69a-cebbe38eea4b None admin] RBAC: Authorizing `identity:create_grant(project_id=cf9744296b5341f08017259b44d86f94, group_id=b9a831cedb0a426580bacc6913f70b8f, role_id=61df19b017494bdcadc9a5e9d8bb9e52)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:03.155247 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f2ba14-24ed-4197-b69a-cebbe38eea4b None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:03.175691 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 201/410] 10.222.0.43 () {58 vars in 1442 bytes} [Tue Sep 6 16:03:02 2022] PUT /identity/v3/projects/cf9744296b5341f08017259b44d86f94/groups/b9a831cedb0a426580bacc6913f70b8f/roles/61df19b017494bdcadc9a5e9d8bb9e52 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:03:03.293904 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e217a4-4f43-4e3c-a60f-a661af55679b None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:03.355545 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:03.415337 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e217a4-4f43-4e3c-a60f-a661af55679b None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:03.449301 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf87be3-ff46-43ad-8660-a34beb6e8564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:03.449941 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf87be3-ff46-43ad-8660-a34beb6e8564 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:03.450218 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf87be3-ff46-43ad-8660-a34beb6e8564 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:03.450523 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf87be3-ff46-43ad-8660-a34beb6e8564 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:03.461665 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf87be3-ff46-43ad-8660-a34beb6e8564 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:03:03.467042 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-8cf87be3-ff46-43ad-8660-a34beb6e8564 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:03:03.467042 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:03.467042 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:03.467042 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:03.467042 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:03.467042 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:03.467042 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:03.467042 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:03.467042 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:03.467042 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:03.467042 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:03.467042 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:03.467042 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:03:03.467042 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:03:03.467042 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:03:03.467042 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:03:03.467989 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:03:03.467989 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:03:03.467989 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:03:03.467989 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:03:03.467989 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:03.467989 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:03.467989 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:03.467989 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:03.467989 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:03.467989 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:03.467989 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:03.467989 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:03.467989 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:03.467989 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:03.467989 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:03.467989 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:03.468797 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:03.468797 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:03.468797 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:03.468797 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:03.468797 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:03:03.468797 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:03:03.468797 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:03:03.468797 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:03:03.468797 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:03:03.468797 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:03:03.468797 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:03:03.468797 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:03.468797 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 210/411] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 6 16:03:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:03.475937 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f2ba14-24ed-4197-b69a-cebbe38eea4b None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:03.558413 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc94abfa-6bc0-45ff-b01e-97a3f94fb7a5 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:03.558737 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc94abfa-6bc0-45ff-b01e-97a3f94fb7a5 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:03.558959 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc94abfa-6bc0-45ff-b01e-97a3f94fb7a5 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:03.560554 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 211/412] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:03:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 16:03:03.567268 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeee355-baec-415e-95a2-c8fe971338ef None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:03.567603 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeee355-baec-415e-95a2-c8fe971338ef None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:03.567952 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeee355-baec-415e-95a2-c8fe971338ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:03.569919 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:03.584045 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f2ba14-24ed-4197-b69a-cebbe38eea4b None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:03.609990 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-cbeee355-baec-415e-95a2-c8fe971338ef None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:03:03.611228 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-cbeee355-baec-415e-95a2-c8fe971338ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:03:03.624957 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe287c0-6233-40a5-a08b-4b065d163fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:03.625622 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe287c0-6233-40a5-a08b-4b065d163fc2 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:03.625906 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe287c0-6233-40a5-a08b-4b065d163fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:03.626175 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe287c0-6233-40a5-a08b-4b065d163fc2 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:03.626852 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe287c0-6233-40a5-a08b-4b065d163fc2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:03.627154 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe287c0-6233-40a5-a08b-4b065d163fc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:03.628597 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe287c0-6233-40a5-a08b-4b065d163fc2 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:03.638854 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 202/413] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 6 16:03:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:03.645293 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe287c0-6233-40a5-a08b-4b065d163fc2 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:03.750220 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 212/414] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:03:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:03:03.799052 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe287c0-6233-40a5-a08b-4b065d163fc2 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:03.838159 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3a1269-7b9f-4230-b361-a3eb1854ef4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:03.838800 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3a1269-7b9f-4230-b361-a3eb1854ef4f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:03.839071 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3a1269-7b9f-4230-b361-a3eb1854ef4f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:03.839347 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3a1269-7b9f-4230-b361-a3eb1854ef4f None admin] PATH_INFO: `/v3/groups/500b3b5ebf3d4b368d6e48864f4b961d` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:03.840441 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3a1269-7b9f-4230-b361-a3eb1854ef4f None admin] RBAC: Authorizing `identity:get_group(group_id=500b3b5ebf3d4b368d6e48864f4b961d)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:03.841997 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3a1269-7b9f-4230-b361-a3eb1854ef4f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:03.844005 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 203/415] 10.222.0.43 () {56 vars in 1180 bytes} [Tue Sep 6 16:03:03 2022] GET /identity/v3/groups/500b3b5ebf3d4b368d6e48864f4b961d => generated 229 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:03.849628 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeee355-baec-415e-95a2-c8fe971338ef None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:03.897269 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:04.010975 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeee355-baec-415e-95a2-c8fe971338ef None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:04.054478 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebb6bae-3644-41ad-adc5-5144f748c9b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:04.055113 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb6bae-3644-41ad-adc5-5144f748c9b1 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:04.055382 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb6bae-3644-41ad-adc5-5144f748c9b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:04.055650 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb6bae-3644-41ad-adc5-5144f748c9b1 None admin] PATH_INFO: `/v3/projects/0ad262a5a8a84066b8eba54c7ea32d39` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:04.061411 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebb6bae-3644-41ad-adc5-5144f748c9b1 None admin] RBAC: Authorizing `identity:get_project(project_id=0ad262a5a8a84066b8eba54c7ea32d39)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:04.062876 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebb6bae-3644-41ad-adc5-5144f748c9b1 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:04.064494 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 213/416] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 6 16:03:03 2022] GET /identity/v3/projects/0ad262a5a8a84066b8eba54c7ea32d39 => generated 344 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:04.070526 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3a1269-7b9f-4230-b361-a3eb1854ef4f None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:04.100977 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:04.157437 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebb6bae-3644-41ad-adc5-5144f748c9b1 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:04.217357 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3a1269-7b9f-4230-b361-a3eb1854ef4f None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:04.267535 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebb6bae-3644-41ad-adc5-5144f748c9b1 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:04.269180 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:04.305719 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb58a03-b72f-4342-9905-8b9b0c5b7edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:04.306554 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb58a03-b72f-4342-9905-8b9b0c5b7edc None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:04.306925 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb58a03-b72f-4342-9905-8b9b0c5b7edc None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:04.307314 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb58a03-b72f-4342-9905-8b9b0c5b7edc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:04.316275 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d0b88b-d77c-4690-840b-183de1446b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:04.317252 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d0b88b-d77c-4690-840b-183de1446b91 None admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:04.317842 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d0b88b-d77c-4690-840b-183de1446b91 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:04.318329 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d0b88b-d77c-4690-840b-183de1446b91 None admin] PATH_INFO: `/v3/projects/0ad262a5a8a84066b8eba54c7ea32d39/groups/500b3b5ebf3d4b368d6e48864f4b961d/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:04.320289 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb58a03-b72f-4342-9905-8b9b0c5b7edc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:03:04.323677 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d0b88b-d77c-4690-840b-183de1446b91 None admin] RBAC: Authorizing `identity:create_grant(project_id=0ad262a5a8a84066b8eba54c7ea32d39, group_id=500b3b5ebf3d4b368d6e48864f4b961d, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:04.326642 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d0b88b-d77c-4690-840b-183de1446b91 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:04.332776 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-dfb58a03-b72f-4342-9905-8b9b0c5b7edc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 16:03:04.332776 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:04.332776 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:04.332776 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:04.332776 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:04.332776 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:04.332776 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:04.332776 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:04.332776 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:04.332776 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:04.332776 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:04.332776 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:04.332776 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:03:04.332776 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:03:04.332776 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:03:04.332776 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:03:04.333754 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:03:04.333754 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:03:04.333754 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:03:04.333754 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:03:04.333754 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:04.333754 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:04.333754 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:04.333754 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:04.333754 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:04.333754 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:04.333754 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:04.333754 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:04.333754 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:04.333754 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:04.333754 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:04.333754 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:04.334629 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:04.334629 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:04.334629 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:04.334629 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:04.334629 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:03:04.334629 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:03:04.334629 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:03:04.334629 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:03:04.334629 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:03:04.334629 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:03:04.334629 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 16:03:04.334629 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:04.334629 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 214/417] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 6 16:03:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:04.339307 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb58a03-b72f-4342-9905-8b9b0c5b7edc None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:04.358768 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 204/418] 10.222.0.43 () {58 vars in 1442 bytes} [Tue Sep 6 16:03:04 2022] PUT /identity/v3/projects/0ad262a5a8a84066b8eba54c7ea32d39/groups/500b3b5ebf3d4b368d6e48864f4b961d/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:03:04.450197 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb58a03-b72f-4342-9905-8b9b0c5b7edc None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:04.489428 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c56e81-b708-4e1f-a34a-9f2978ba3f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:04.490080 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c56e81-b708-4e1f-a34a-9f2978ba3f36 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:04.490356 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c56e81-b708-4e1f-a34a-9f2978ba3f36 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:04.490657 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c56e81-b708-4e1f-a34a-9f2978ba3f36 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:04.491338 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c56e81-b708-4e1f-a34a-9f2978ba3f36 None admin] RBAC: Adding query filter params (name=member) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:04.491636 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c56e81-b708-4e1f-a34a-9f2978ba3f36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:04.493070 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c56e81-b708-4e1f-a34a-9f2978ba3f36 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:04.501950 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 215/419] 10.222.0.43 () {56 vars in 1101 bytes} [Tue Sep 6 16:03:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:04.508006 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d0b88b-d77c-4690-840b-183de1446b91 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:04.621908 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:04.655288 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d0b88b-d77c-4690-840b-183de1446b91 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:04.706777 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a968fbc2-8632-48e7-9ecc-cc9a2aa10b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:04.707391 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a968fbc2-8632-48e7-9ecc-cc9a2aa10b20 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:04.707667 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a968fbc2-8632-48e7-9ecc-cc9a2aa10b20 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:04.707935 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a968fbc2-8632-48e7-9ecc-cc9a2aa10b20 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:04.713365 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea81ecb-1b27-41e6-863a-72edfff8160c None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:04.713920 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea81ecb-1b27-41e6-863a-72edfff8160c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:04.714416 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea81ecb-1b27-41e6-863a-72edfff8160c None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:04.716413 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 216/420] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:03:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:03:04.722712 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6456384d-cb68-4c9f-a010-59aa877f1830 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:04.723128 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6456384d-cb68-4c9f-a010-59aa877f1830 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:04.723558 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6456384d-cb68-4c9f-a010-59aa877f1830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:04.745393 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-6456384d-cb68-4c9f-a010-59aa877f1830 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:03:04.747006 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-6456384d-cb68-4c9f-a010-59aa877f1830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:03:04.767074 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:04.782101 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a968fbc2-8632-48e7-9ecc-cc9a2aa10b20 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:04.783639 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a968fbc2-8632-48e7-9ecc-cc9a2aa10b20 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:04.829525 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 217/421] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:03:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:03:04.836678 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f09c10-c064-4972-91cf-4269ad46b008 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:04.837255 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f09c10-c064-4972-91cf-4269ad46b008 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:04.837727 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f09c10-c064-4972-91cf-4269ad46b008 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:04.839776 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 218/422] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:03:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 16:03:04.844596 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf477e21-f122-4199-8479-f52c43ba1bf5 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:04.844926 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf477e21-f122-4199-8479-f52c43ba1bf5 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:04.845204 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf477e21-f122-4199-8479-f52c43ba1bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:04.863780 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:04.864427 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-bf477e21-f122-4199-8479-f52c43ba1bf5 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:03:04.865906 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-bf477e21-f122-4199-8479-f52c43ba1bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:03:04.892192 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 205/423] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 6 16:03:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 16:03:04.899334 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a968fbc2-8632-48e7-9ecc-cc9a2aa10b20 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:04.989175 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 219/424] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:03:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 16:03:05.030921 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a968fbc2-8632-48e7-9ecc-cc9a2aa10b20 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:05.076266 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5649f93d-75af-4964-b2bb-115b4e45ccbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:05.076966 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5649f93d-75af-4964-b2bb-115b4e45ccbc None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:05.077240 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5649f93d-75af-4964-b2bb-115b4e45ccbc None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:05.077508 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5649f93d-75af-4964-b2bb-115b4e45ccbc None admin] PATH_INFO: `/v3/projects/5e7a87d573a8446bbcbf9e28149a392f` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:05.080079 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5649f93d-75af-4964-b2bb-115b4e45ccbc None admin] RBAC: Authorizing `identity:get_project(project_id=5e7a87d573a8446bbcbf9e28149a392f)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:05.082007 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5649f93d-75af-4964-b2bb-115b4e45ccbc None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:05.083942 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 206/425] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 6 16:03:04 2022] GET /identity/v3/projects/5e7a87d573a8446bbcbf9e28149a392f => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:03:05.089806 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf477e21-f122-4199-8479-f52c43ba1bf5 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:05.150011 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5649f93d-75af-4964-b2bb-115b4e45ccbc None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:05.226328 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:05.257122 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:05.259761 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf477e21-f122-4199-8479-f52c43ba1bf5 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:05.309903 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5649f93d-75af-4964-b2bb-115b4e45ccbc None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:05.334894 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62351c64-a22c-4f44-9c13-c82bb3c9d51b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:05.335540 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62351c64-a22c-4f44-9c13-c82bb3c9d51b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:05.335815 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62351c64-a22c-4f44-9c13-c82bb3c9d51b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:05.336085 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62351c64-a22c-4f44-9c13-c82bb3c9d51b None admin] PATH_INFO: `/v3/groups/b9a831cedb0a426580bacc6913f70b8f` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:05.341415 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62351c64-a22c-4f44-9c13-c82bb3c9d51b None admin] RBAC: Authorizing `identity:get_group(group_id=b9a831cedb0a426580bacc6913f70b8f)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:05.342894 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62351c64-a22c-4f44-9c13-c82bb3c9d51b None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:05.344663 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 220/426] 10.222.0.43 () {56 vars in 1180 bytes} [Tue Sep 6 16:03:05 2022] GET /identity/v3/groups/b9a831cedb0a426580bacc6913f70b8f => generated 226 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:05.345384 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62351c64-a22c-4f44-9c13-c82bb3c9d51b None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:05.345921 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1af1fc-ba81-4a56-bd08-e498c04b31e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:05.346567 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1af1fc-ba81-4a56-bd08-e498c04b31e8 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:05.346847 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1af1fc-ba81-4a56-bd08-e498c04b31e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:05.347121 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1af1fc-ba81-4a56-bd08-e498c04b31e8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:05.359632 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1af1fc-ba81-4a56-bd08-e498c04b31e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:03:05.365007 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-eb1af1fc-ba81-4a56-bd08-e498c04b31e8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 16:03:05.365007 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:05.365007 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:05.365007 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:05.365007 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:05.365007 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:05.365007 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:05.365007 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:05.365007 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:05.365007 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:05.365007 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:05.365007 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:05.365007 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:03:05.365007 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:03:05.365007 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:03:05.365007 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:03:05.365885 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:03:05.365885 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:03:05.365885 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:03:05.365885 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:03:05.365885 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:05.365885 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:05.365885 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:05.365885 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:05.365885 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:05.365885 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:05.365885 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:05.365885 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:05.365885 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:05.365885 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:05.365885 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:05.365885 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:05.366674 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:05.366674 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:05.366674 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:05.366674 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:05.366674 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:03:05.366674 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:03:05.366674 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:03:05.366674 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:03:05.366674 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:03:05.366674 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:03:05.366674 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 16:03:05.366674 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:05.366674 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 207/427] 10.222.0.43 () {56 vars in 1114 bytes} [Tue Sep 6 16:03:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:03:05.367354 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1af1fc-ba81-4a56-bd08-e498c04b31e8 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:05.458983 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62351c64-a22c-4f44-9c13-c82bb3c9d51b None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:05.484292 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1af1fc-ba81-4a56-bd08-e498c04b31e8 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:05.495209 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:05.505584 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50410942-8e05-4f97-8361-521885b42841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:05.506321 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50410942-8e05-4f97-8361-521885b42841 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:05.506598 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50410942-8e05-4f97-8361-521885b42841 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:05.507122 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50410942-8e05-4f97-8361-521885b42841 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:05.521243 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50410942-8e05-4f97-8361-521885b42841 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:03:05.527154 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ccb57f-8770-4285-b91d-fd42b4783a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:05.527574 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-50410942-8e05-4f97-8361-521885b42841 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 16:03:05.527574 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:05.527574 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:05.527574 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:05.527574 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:05.527574 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:05.527574 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:05.527574 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:05.527574 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:05.527574 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:05.527574 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:05.527574 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:05.527574 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:03:05.527574 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:03:05.527574 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:03:05.527574 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:03:05.528414 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:03:05.528414 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:03:05.528414 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:03:05.528414 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:03:05.528414 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:05.528414 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:05.528414 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:05.528414 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:05.528414 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:05.528414 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:05.528414 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:05.528414 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:05.528414 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:05.528414 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:05.528414 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:05.528414 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:05.529239 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:05.529239 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:05.529239 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:05.529239 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:05.529239 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:03:05.529239 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:03:05.529239 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:03:05.529239 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:03:05.529239 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:03:05.529239 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:03:05.529239 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 16:03:05.529239 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:05.529834 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ccb57f-8770-4285-b91d-fd42b4783a7b None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:05.529834 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ccb57f-8770-4285-b91d-fd42b4783a7b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:05.529834 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ccb57f-8770-4285-b91d-fd42b4783a7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:05.530015 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 221/428] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 6 16:03:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:05.530084 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ccb57f-8770-4285-b91d-fd42b4783a7b None admin] RBAC: Adding query filter params (group.id=b9a831cedb0a426580bacc6913f70b8f, role.id=15252b0b89894c1e8a6bd9a98b1c3ab7, scope.project.id=5e7a87d573a8446bbcbf9e28149a392f) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:05.530155 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50410942-8e05-4f97-8361-521885b42841 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:05.530219 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ccb57f-8770-4285-b91d-fd42b4783a7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:05.531809 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ccb57f-8770-4285-b91d-fd42b4783a7b None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:05.540221 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 208/429] 10.222.0.43 () {56 vars in 1376 bytes} [Tue Sep 6 16:03:05 2022] GET /identity/v3/role_assignments?group.id=b9a831cedb0a426580bacc6913f70b8f&scope.project.id=5e7a87d573a8446bbcbf9e28149a392f&role.id=15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 611 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:05.541188 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ccb57f-8770-4285-b91d-fd42b4783a7b None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:05.548918 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:05.660542 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50410942-8e05-4f97-8361-521885b42841 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:05.667877 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ccb57f-8770-4285-b91d-fd42b4783a7b None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:05.696976 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb831f9-12d5-43ab-b490-0ef47bc95d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:05.697594 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb831f9-12d5-43ab-b490-0ef47bc95d7e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:05.697875 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb831f9-12d5-43ab-b490-0ef47bc95d7e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:05.698149 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb831f9-12d5-43ab-b490-0ef47bc95d7e None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:05.698831 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb831f9-12d5-43ab-b490-0ef47bc95d7e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:05.699127 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb831f9-12d5-43ab-b490-0ef47bc95d7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:05.700554 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb831f9-12d5-43ab-b490-0ef47bc95d7e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:05.705204 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af16212-b745-49da-84a0-d4726b9a4118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:05.706183 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af16212-b745-49da-84a0-d4726b9a4118 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:05.706621 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af16212-b745-49da-84a0-d4726b9a4118 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:05.707094 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af16212-b745-49da-84a0-d4726b9a4118 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:05.707928 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af16212-b745-49da-84a0-d4726b9a4118 None admin] RBAC: Adding query filter params (name=member) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:05.708483 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af16212-b745-49da-84a0-d4726b9a4118 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:05.710205 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af16212-b745-49da-84a0-d4726b9a4118 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:05.710769 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 222/430] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 6 16:03:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:05.712014 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9242ec-8d4c-4d71-8969-a9ae099fffdb None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:05.712300 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9242ec-8d4c-4d71-8969-a9ae099fffdb None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:05.712694 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9242ec-8d4c-4d71-8969-a9ae099fffdb None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:05.714030 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 223/431] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:03:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:03:05.716815 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9242ec-8d4c-4d71-8969-a9ae099fffdb None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:05.724239 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 209/432] 10.222.0.43 () {56 vars in 1101 bytes} [Tue Sep 6 16:03:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:05.725459 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b5c580-9af4-488c-968b-0e69ab40365f None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:05.725747 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b5c580-9af4-488c-968b-0e69ab40365f None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:05.726028 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b5c580-9af4-488c-968b-0e69ab40365f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:05.746329 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-48b5c580-9af4-488c-968b-0e69ab40365f None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:03:05.747508 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-48b5c580-9af4-488c-968b-0e69ab40365f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:03:05.777405 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:05.826029 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 210/433] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:03:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:03:05.826416 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:05.827541 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b5c580-9af4-488c-968b-0e69ab40365f None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:05.870924 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9242ec-8d4c-4d71-8969-a9ae099fffdb None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:05.916293 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f961bfa2-0bea-4812-94c3-9a7d6e832b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:05.917032 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f961bfa2-0bea-4812-94c3-9a7d6e832b2e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:05.917327 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f961bfa2-0bea-4812-94c3-9a7d6e832b2e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:05.917597 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f961bfa2-0bea-4812-94c3-9a7d6e832b2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:05.948197 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b5c580-9af4-488c-968b-0e69ab40365f None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:05.996018 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f961bfa2-0bea-4812-94c3-9a7d6e832b2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:05.996324 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455ce1c2-5fbc-4f61-a5af-0540097bb7c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:05.997756 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ce1c2-5fbc-4f61-a5af-0540097bb7c7 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:05.997756 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ce1c2-5fbc-4f61-a5af-0540097bb7c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:05.997756 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ce1c2-5fbc-4f61-a5af-0540097bb7c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:05.999275 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f961bfa2-0bea-4812-94c3-9a7d6e832b2e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:06.038459 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:06.086816 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455ce1c2-5fbc-4f61-a5af-0540097bb7c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:06.088564 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455ce1c2-5fbc-4f61-a5af-0540097bb7c7 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:06.097838 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:06.128274 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 224/434] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 6 16:03:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 16:03:06.130257 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f961bfa2-0bea-4812-94c3-9a7d6e832b2e None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:06.222333 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 211/435] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 6 16:03:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 16:03:06.223005 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455ce1c2-5fbc-4f61-a5af-0540097bb7c7 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:06.258282 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f961bfa2-0bea-4812-94c3-9a7d6e832b2e None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:06.291891 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022c607e-08e8-4750-b5c8-70c06c6488f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:06.292451 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022c607e-08e8-4750-b5c8-70c06c6488f5 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:06.292727 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022c607e-08e8-4750-b5c8-70c06c6488f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:06.293025 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022c607e-08e8-4750-b5c8-70c06c6488f5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:06.300909 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:06.304249 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022c607e-08e8-4750-b5c8-70c06c6488f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:03:06.309105 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-022c607e-08e8-4750-b5c8-70c06c6488f5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 16:03:06.309105 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:06.309105 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:06.309105 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:06.309105 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:06.309105 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:06.309105 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:06.309105 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:06.309105 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:06.309105 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:06.309105 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:06.309105 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:06.309105 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:03:06.309105 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:03:06.309105 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:03:06.309105 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:03:06.309988 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:03:06.309988 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:03:06.309988 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:03:06.309988 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:03:06.309988 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:06.309988 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:06.309988 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:06.309988 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:06.309988 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:06.309988 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:06.309988 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:06.309988 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:06.309988 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:06.309988 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:06.309988 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:06.309988 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:06.310835 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:06.310835 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:06.310835 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:06.310835 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:06.310835 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:03:06.310835 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:03:06.310835 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:03:06.310835 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:03:06.310835 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:03:06.310835 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:03:06.310835 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 16:03:06.310835 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:06.310835 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 225/436] 10.222.0.43 () {56 vars in 1114 bytes} [Tue Sep 6 16:03:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:06.311502 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022c607e-08e8-4750-b5c8-70c06c6488f5 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:06.347028 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455ce1c2-5fbc-4f61-a5af-0540097bb7c7 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:06.349526 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:06.391159 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09ab2e3-c3ca-456a-a1f6-7e91a68b577c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:06.391799 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09ab2e3-c3ca-456a-a1f6-7e91a68b577c None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:06.392075 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09ab2e3-c3ca-456a-a1f6-7e91a68b577c None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:06.392346 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09ab2e3-c3ca-456a-a1f6-7e91a68b577c None admin] PATH_INFO: `/v3/projects/5e7a87d573a8446bbcbf9e28149a392f` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:06.393490 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09ab2e3-c3ca-456a-a1f6-7e91a68b577c None admin] RBAC: Authorizing `identity:get_project(project_id=5e7a87d573a8446bbcbf9e28149a392f)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:06.394952 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09ab2e3-c3ca-456a-a1f6-7e91a68b577c None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:06.401400 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 212/437] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 6 16:03:06 2022] GET /identity/v3/projects/5e7a87d573a8446bbcbf9e28149a392f => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:06.401400 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09ab2e3-c3ca-456a-a1f6-7e91a68b577c None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:06.424877 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022c607e-08e8-4750-b5c8-70c06c6488f5 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:06.461363 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cd28bc-fb4c-43a2-8939-adc8cf795ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:06.461997 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd28bc-fb4c-43a2-8939-adc8cf795ad6 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:06.462286 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd28bc-fb4c-43a2-8939-adc8cf795ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:06.462548 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd28bc-fb4c-43a2-8939-adc8cf795ad6 None admin] PATH_INFO: `/v3/projects/cf9744296b5341f08017259b44d86f94` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:06.463651 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cd28bc-fb4c-43a2-8939-adc8cf795ad6 None admin] RBAC: Authorizing `identity:get_project(project_id=cf9744296b5341f08017259b44d86f94)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:06.465137 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cd28bc-fb4c-43a2-8939-adc8cf795ad6 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:06.471569 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 226/438] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 6 16:03:06 2022] GET /identity/v3/projects/cf9744296b5341f08017259b44d86f94 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:06.471569 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cd28bc-fb4c-43a2-8939-adc8cf795ad6 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:06.506985 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09ab2e3-c3ca-456a-a1f6-7e91a68b577c None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:06.538369 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:06.546002 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da6245f-00fa-4560-b465-7fe923b9b836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:06.546615 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da6245f-00fa-4560-b465-7fe923b9b836 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:06.546890 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da6245f-00fa-4560-b465-7fe923b9b836 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:06.547156 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da6245f-00fa-4560-b465-7fe923b9b836 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:06.547816 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da6245f-00fa-4560-b465-7fe923b9b836 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:06.548111 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da6245f-00fa-4560-b465-7fe923b9b836 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:06.549982 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da6245f-00fa-4560-b465-7fe923b9b836 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:06.563114 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 213/439] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 6 16:03:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:06.563114 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da6245f-00fa-4560-b465-7fe923b9b836 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:06.573097 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cd28bc-fb4c-43a2-8939-adc8cf795ad6 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:06.574791 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:06.610363 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40927e4c-9128-469f-928c-87ecf913b382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:06.610999 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40927e4c-9128-469f-928c-87ecf913b382 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:06.611277 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40927e4c-9128-469f-928c-87ecf913b382 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:06.611539 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40927e4c-9128-469f-928c-87ecf913b382 None admin] PATH_INFO: `/v3/groups/b9a831cedb0a426580bacc6913f70b8f` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:06.612666 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40927e4c-9128-469f-928c-87ecf913b382 None admin] RBAC: Authorizing `identity:get_group(group_id=b9a831cedb0a426580bacc6913f70b8f)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:06.614133 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40927e4c-9128-469f-928c-87ecf913b382 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:06.620527 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 227/440] 10.222.0.43 () {56 vars in 1180 bytes} [Tue Sep 6 16:03:06 2022] GET /identity/v3/groups/b9a831cedb0a426580bacc6913f70b8f => generated 226 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:06.620527 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40927e4c-9128-469f-928c-87ecf913b382 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:06.669746 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da6245f-00fa-4560-b465-7fe923b9b836 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:06.706566 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e16aba5-673a-453b-8acf-0ae8c14a2e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:06.707208 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e16aba5-673a-453b-8acf-0ae8c14a2e84 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:06.707485 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e16aba5-673a-453b-8acf-0ae8c14a2e84 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:06.707762 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e16aba5-673a-453b-8acf-0ae8c14a2e84 None admin] PATH_INFO: `/v3/groups/b9a831cedb0a426580bacc6913f70b8f` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:06.708957 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e16aba5-673a-453b-8acf-0ae8c14a2e84 None admin] RBAC: Authorizing `identity:get_group(group_id=b9a831cedb0a426580bacc6913f70b8f)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:06.710419 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e16aba5-673a-453b-8acf-0ae8c14a2e84 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:06.712797 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 214/441] 10.222.0.43 () {56 vars in 1180 bytes} [Tue Sep 6 16:03:06 2022] GET /identity/v3/groups/b9a831cedb0a426580bacc6913f70b8f => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:06.712797 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e16aba5-673a-453b-8acf-0ae8c14a2e84 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:06.733513 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40927e4c-9128-469f-928c-87ecf913b382 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:06.770343 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e808ed0c-6d0f-4ccb-8445-d2dc7db5252b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:06.770982 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e808ed0c-6d0f-4ccb-8445-d2dc7db5252b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:06.771259 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e808ed0c-6d0f-4ccb-8445-d2dc7db5252b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:06.771529 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e808ed0c-6d0f-4ccb-8445-d2dc7db5252b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:06.782097 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:06.814381 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:06.828683 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e16aba5-673a-453b-8acf-0ae8c14a2e84 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:06.844404 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e808ed0c-6d0f-4ccb-8445-d2dc7db5252b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:06.845890 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e808ed0c-6d0f-4ccb-8445-d2dc7db5252b None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:06.861796 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b45d70-1594-40e6-99d3-b0edf542c964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:06.862394 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b45d70-1594-40e6-99d3-b0edf542c964 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:06.862677 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b45d70-1594-40e6-99d3-b0edf542c964 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:06.862943 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b45d70-1594-40e6-99d3-b0edf542c964 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:06.863667 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b45d70-1594-40e6-99d3-b0edf542c964 None admin] RBAC: Adding query filter params (group.id=b9a831cedb0a426580bacc6913f70b8f, role.id=61df19b017494bdcadc9a5e9d8bb9e52, scope.project.id=5e7a87d573a8446bbcbf9e28149a392f) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:06.863983 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b45d70-1594-40e6-99d3-b0edf542c964 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:06.865518 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b45d70-1594-40e6-99d3-b0edf542c964 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:06.871087 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 215/442] 10.222.0.43 () {56 vars in 1376 bytes} [Tue Sep 6 16:03:06 2022] GET /identity/v3/role_assignments?group.id=b9a831cedb0a426580bacc6913f70b8f&scope.project.id=5e7a87d573a8446bbcbf9e28149a392f&role.id=61df19b017494bdcadc9a5e9d8bb9e52 => generated 611 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:06.871855 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b45d70-1594-40e6-99d3-b0edf542c964 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:06.968342 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 228/443] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 6 16:03:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 16:03:06.969853 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44201dc0-e273-4444-b2d6-87f44a5d5cb4 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:06.970203 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44201dc0-e273-4444-b2d6-87f44a5d5cb4 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:06.970545 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44201dc0-e273-4444-b2d6-87f44a5d5cb4 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:06.972064 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 229/444] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:03:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:03:06.976986 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44201dc0-e273-4444-b2d6-87f44a5d5cb4 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:06.992850 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b45d70-1594-40e6-99d3-b0edf542c964 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:07.020125 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:07.034079 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa5e419-e29e-4bd3-b6d1-aebc81a24aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:07.037562 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5e419-e29e-4bd3-b6d1-aebc81a24aea None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:07.037562 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5e419-e29e-4bd3-b6d1-aebc81a24aea None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:07.037562 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5e419-e29e-4bd3-b6d1-aebc81a24aea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:07.037562 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa5e419-e29e-4bd3-b6d1-aebc81a24aea None admin] RBAC: Adding query filter params (group.id=b9a831cedb0a426580bacc6913f70b8f, role.id=15252b0b89894c1e8a6bd9a98b1c3ab7, scope.project.id=cf9744296b5341f08017259b44d86f94) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:07.037562 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa5e419-e29e-4bd3-b6d1-aebc81a24aea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:07.037910 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa5e419-e29e-4bd3-b6d1-aebc81a24aea None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:07.044338 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 216/445] 10.222.0.43 () {56 vars in 1376 bytes} [Tue Sep 6 16:03:06 2022] GET /identity/v3/role_assignments?group.id=b9a831cedb0a426580bacc6913f70b8f&scope.project.id=cf9744296b5341f08017259b44d86f94&role.id=15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 611 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:07.045736 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d1c06-1137-4bd9-a4e2-0a7dd9d7d430 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:07.046177 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d1c06-1137-4bd9-a4e2-0a7dd9d7d430 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:07.046590 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d1c06-1137-4bd9-a4e2-0a7dd9d7d430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:07.067283 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-566d1c06-1137-4bd9-a4e2-0a7dd9d7d430 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:03:07.069024 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-566d1c06-1137-4bd9-a4e2-0a7dd9d7d430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:03:07.078853 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:07.138363 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44201dc0-e273-4444-b2d6-87f44a5d5cb4 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:07.173206 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 217/446] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:03:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:03:07.183338 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426e2e4f-4263-4d62-9926-105b3da29a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:07.183991 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426e2e4f-4263-4d62-9926-105b3da29a98 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:07.184260 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426e2e4f-4263-4d62-9926-105b3da29a98 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:07.184525 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426e2e4f-4263-4d62-9926-105b3da29a98 None admin] PATH_INFO: `/v3/projects/cf9744296b5341f08017259b44d86f94` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:07.185768 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426e2e4f-4263-4d62-9926-105b3da29a98 None admin] RBAC: Authorizing `identity:get_project(project_id=cf9744296b5341f08017259b44d86f94)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:07.188280 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426e2e4f-4263-4d62-9926-105b3da29a98 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:07.193777 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 230/447] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 6 16:03:06 2022] GET /identity/v3/projects/cf9744296b5341f08017259b44d86f94 => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:07.194919 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566d1c06-1137-4bd9-a4e2-0a7dd9d7d430 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:07.307786 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566d1c06-1137-4bd9-a4e2-0a7dd9d7d430 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:07.326253 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:07.344341 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804c9005-11f7-4dcd-b668-db0ffb0b163f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:07.345011 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804c9005-11f7-4dcd-b668-db0ffb0b163f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:07.345287 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804c9005-11f7-4dcd-b668-db0ffb0b163f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:07.345554 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804c9005-11f7-4dcd-b668-db0ffb0b163f None admin] PATH_INFO: `/v3/groups/b9a831cedb0a426580bacc6913f70b8f` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:07.346663 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804c9005-11f7-4dcd-b668-db0ffb0b163f None admin] RBAC: Authorizing `identity:get_group(group_id=b9a831cedb0a426580bacc6913f70b8f)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:07.348119 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804c9005-11f7-4dcd-b668-db0ffb0b163f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:07.349804 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 218/448] 10.222.0.43 () {56 vars in 1180 bytes} [Tue Sep 6 16:03:07 2022] GET /identity/v3/groups/b9a831cedb0a426580bacc6913f70b8f => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:07.355755 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426e2e4f-4263-4d62-9926-105b3da29a98 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:07.444725 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804c9005-11f7-4dcd-b668-db0ffb0b163f None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:07.475231 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426e2e4f-4263-4d62-9926-105b3da29a98 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:07.515961 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62efa8aa-f3b5-42ca-bf62-4a75c0796cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:07.516569 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62efa8aa-f3b5-42ca-bf62-4a75c0796cb9 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:07.516882 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62efa8aa-f3b5-42ca-bf62-4a75c0796cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:07.517151 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62efa8aa-f3b5-42ca-bf62-4a75c0796cb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:07.517868 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62efa8aa-f3b5-42ca-bf62-4a75c0796cb9 None admin] RBAC: Adding query filter params (group.id=b9a831cedb0a426580bacc6913f70b8f, role.id=61df19b017494bdcadc9a5e9d8bb9e52, scope.project.id=cf9744296b5341f08017259b44d86f94) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:07.518180 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62efa8aa-f3b5-42ca-bf62-4a75c0796cb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:07.519612 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62efa8aa-f3b5-42ca-bf62-4a75c0796cb9 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:07.525439 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 231/449] 10.222.0.43 () {56 vars in 1376 bytes} [Tue Sep 6 16:03:07 2022] GET /identity/v3/role_assignments?group.id=b9a831cedb0a426580bacc6913f70b8f&scope.project.id=cf9744296b5341f08017259b44d86f94&role.id=61df19b017494bdcadc9a5e9d8bb9e52 => generated 611 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:07.555050 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804c9005-11f7-4dcd-b668-db0ffb0b163f None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:07.593544 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b342387f-6e35-4bd7-9d98-b631f42d0d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:07.594113 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b342387f-6e35-4bd7-9d98-b631f42d0d07 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:07.594385 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b342387f-6e35-4bd7-9d98-b631f42d0d07 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:07.594653 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b342387f-6e35-4bd7-9d98-b631f42d0d07 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:07.606561 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b342387f-6e35-4bd7-9d98-b631f42d0d07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:03:07.611472 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-b342387f-6e35-4bd7-9d98-b631f42d0d07 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:03:07.611472 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:07.611472 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:07.611472 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:07.611472 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:07.611472 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:07.611472 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:07.611472 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:07.611472 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:07.611472 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:07.611472 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:07.611472 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:07.611472 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:03:07.611472 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:03:07.611472 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:03:07.611472 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:03:07.612486 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:03:07.612486 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:03:07.612486 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:03:07.612486 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:03:07.612486 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:07.612486 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:07.612486 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:07.612486 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:07.612486 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:07.612486 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:07.612486 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:07.612486 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:07.612486 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:07.612486 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:07.612486 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:07.612486 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:07.613538 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:07.613538 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:07.613538 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:07.613538 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:07.613538 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:03:07.613538 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:03:07.613538 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:03:07.613538 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:03:07.613538 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:03:07.613538 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:03:07.613538 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:03:07.613538 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:07.613538 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 219/450] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 6 16:03:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:03:07.618997 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62efa8aa-f3b5-42ca-bf62-4a75c0796cb9 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:07.622212 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:07.729973 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62efa8aa-f3b5-42ca-bf62-4a75c0796cb9 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:07.764264 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76a1703-5919-4024-81e8-aa6933bab163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:07.765443 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76a1703-5919-4024-81e8-aa6933bab163 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:07.765713 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76a1703-5919-4024-81e8-aa6933bab163 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:07.765977 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76a1703-5919-4024-81e8-aa6933bab163 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:07.766633 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76a1703-5919-4024-81e8-aa6933bab163 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:07.766928 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76a1703-5919-4024-81e8-aa6933bab163 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:07.768283 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76a1703-5919-4024-81e8-aa6933bab163 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:07.780744 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 232/451] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 6 16:03:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:07.788422 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b342387f-6e35-4bd7-9d98-b631f42d0d07 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:07.836099 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:07.888322 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b342387f-6e35-4bd7-9d98-b631f42d0d07 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:07.923290 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcf2bc5-cf39-4113-9f8b-fa2fb47c3d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:07.923884 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf2bc5-cf39-4113-9f8b-fa2fb47c3d1b None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:07.924156 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf2bc5-cf39-4113-9f8b-fa2fb47c3d1b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:07.924416 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf2bc5-cf39-4113-9f8b-fa2fb47c3d1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:08.000014 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcf2bc5-cf39-4113-9f8b-fa2fb47c3d1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:08.001600 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcf2bc5-cf39-4113-9f8b-fa2fb47c3d1b None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:08.045424 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:08.119373 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 220/452] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 6 16:03:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 16:03:08.127575 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76a1703-5919-4024-81e8-aa6933bab163 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:08.192246 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:08.233286 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76a1703-5919-4024-81e8-aa6933bab163 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:08.272700 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1ac85c-ddae-4e28-88ae-652ba873655f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:08.273369 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1ac85c-ddae-4e28-88ae-652ba873655f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:08.273643 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1ac85c-ddae-4e28-88ae-652ba873655f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:08.273908 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1ac85c-ddae-4e28-88ae-652ba873655f None admin] PATH_INFO: `/v3/projects/0ad262a5a8a84066b8eba54c7ea32d39` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:08.275174 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1ac85c-ddae-4e28-88ae-652ba873655f None admin] RBAC: Authorizing `identity:get_project(project_id=0ad262a5a8a84066b8eba54c7ea32d39)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:08.276672 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1ac85c-ddae-4e28-88ae-652ba873655f None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:08.278357 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 233/453] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 6 16:03:08 2022] GET /identity/v3/projects/0ad262a5a8a84066b8eba54c7ea32d39 => generated 344 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:08.283982 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcf2bc5-cf39-4113-9f8b-fa2fb47c3d1b None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:08.385424 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcf2bc5-cf39-4113-9f8b-fa2fb47c3d1b None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:08.419444 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0233ee-8c64-4040-9c85-10a8b65b337e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:08.420051 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0233ee-8c64-4040-9c85-10a8b65b337e None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:08.420324 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0233ee-8c64-4040-9c85-10a8b65b337e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:08.420585 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0233ee-8c64-4040-9c85-10a8b65b337e None admin] PATH_INFO: `/v3/groups/500b3b5ebf3d4b368d6e48864f4b961d` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:08.421824 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0233ee-8c64-4040-9c85-10a8b65b337e None admin] RBAC: Authorizing `identity:get_group(group_id=500b3b5ebf3d4b368d6e48864f4b961d)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:08.423282 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0233ee-8c64-4040-9c85-10a8b65b337e None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:08.424821 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 221/454] 10.222.0.43 () {56 vars in 1180 bytes} [Tue Sep 6 16:03:08 2022] GET /identity/v3/groups/500b3b5ebf3d4b368d6e48864f4b961d => generated 229 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:08.431182 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1ac85c-ddae-4e28-88ae-652ba873655f None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:08.534224 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1ac85c-ddae-4e28-88ae-652ba873655f None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:08.563736 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:08.573494 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9899ea0-7036-4a3e-b1e5-ba73a6fb89ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:08.574402 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9899ea0-7036-4a3e-b1e5-ba73a6fb89ca None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:08.574733 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9899ea0-7036-4a3e-b1e5-ba73a6fb89ca None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:08.575103 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9899ea0-7036-4a3e-b1e5-ba73a6fb89ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:08.576233 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9899ea0-7036-4a3e-b1e5-ba73a6fb89ca None admin] RBAC: Adding query filter params (group.id=500b3b5ebf3d4b368d6e48864f4b961d, role.id=edaabdea8be84a7980fb2c651fe52f63, scope.project.id=0ad262a5a8a84066b8eba54c7ea32d39) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:08.576642 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9899ea0-7036-4a3e-b1e5-ba73a6fb89ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:08.578915 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9899ea0-7036-4a3e-b1e5-ba73a6fb89ca None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:08.587219 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 234/455] 10.222.0.43 () {56 vars in 1376 bytes} [Tue Sep 6 16:03:08 2022] GET /identity/v3/role_assignments?group.id=500b3b5ebf3d4b368d6e48864f4b961d&scope.project.id=0ad262a5a8a84066b8eba54c7ea32d39&role.id=edaabdea8be84a7980fb2c651fe52f63 => generated 611 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:03:11.021540 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2574822b-fa08-4ece-8882-1eb5d1fece39 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:11.022100 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2574822b-fa08-4ece-8882-1eb5d1fece39 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:11.022100 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2574822b-fa08-4ece-8882-1eb5d1fece39 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:11.023435 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 222/456] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:03:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:03:11.028726 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9e02d2-ca65-49f9-94d9-8437ed818e80 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:11.029577 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9e02d2-ca65-49f9-94d9-8437ed818e80 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:11.030070 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9e02d2-ca65-49f9-94d9-8437ed818e80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:11.057799 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-ac9e02d2-ca65-49f9-94d9-8437ed818e80 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:03:11.059373 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ac9e02d2-ca65-49f9-94d9-8437ed818e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:03:11.062754 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65749e50-ac7a-41c0-815e-05db7f31ef1a None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:11.063038 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65749e50-ac7a-41c0-815e-05db7f31ef1a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:11.063306 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65749e50-ac7a-41c0-815e-05db7f31ef1a None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:11.065023 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 223/457] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:03:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:03:11.071322 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9921451f-4125-4b3d-ba3f-abbd7b32e871 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:11.071322 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9921451f-4125-4b3d-ba3f-abbd7b32e871 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:11.071322 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9921451f-4125-4b3d-ba3f-abbd7b32e871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:11.094906 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-9921451f-4125-4b3d-ba3f-abbd7b32e871 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:03:11.094906 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-9921451f-4125-4b3d-ba3f-abbd7b32e871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:03:11.125128 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:11.186218 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 235/458] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:03:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:03:11.202872 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 224/459] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:03:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 16:03:11.396267 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ba6fc-cb94-4eac-a948-3cce9c872fd4 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:11.396624 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ba6fc-cb94-4eac-a948-3cce9c872fd4 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:11.396855 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ba6fc-cb94-4eac-a948-3cce9c872fd4 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:11.398347 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 236/460] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:03:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:03:11.403599 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5efb8b5-d096-45b4-bf15-a531ef125251 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:11.403884 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5efb8b5-d096-45b4-bf15-a531ef125251 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:11.404155 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5efb8b5-d096-45b4-bf15-a531ef125251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:11.426837 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-f5efb8b5-d096-45b4-bf15-a531ef125251 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:03:11.428032 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f5efb8b5-d096-45b4-bf15-a531ef125251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:03:11.481098 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192ba6fc-cb94-4eac-a948-3cce9c872fd4 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:11.513110 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 225/461] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:03:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:03:11.613335 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5efb8b5-d096-45b4-bf15-a531ef125251 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:11.627574 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192ba6fc-cb94-4eac-a948-3cce9c872fd4 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:11.674357 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:11.684418 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6d7537-d44f-49e7-8bfa-393397632fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:11.685093 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6d7537-d44f-49e7-8bfa-393397632fb9 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:11.685369 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6d7537-d44f-49e7-8bfa-393397632fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:11.685639 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6d7537-d44f-49e7-8bfa-393397632fb9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:11.692919 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:11.697995 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6d7537-d44f-49e7-8bfa-393397632fb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:11.699635 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6d7537-d44f-49e7-8bfa-393397632fb9 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:11.715825 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-bc6d7537-d44f-49e7-8bfa-393397632fb9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:11.715825 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:11.715825 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:11.715825 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:11.715825 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:11.715825 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:11.715825 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:11.715825 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:11.715825 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:11.715825 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:11.715825 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:11.715825 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:11.715825 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:03:11.715825 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:03:11.715825 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:03:11.715825 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:03:11.716961 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:11.716961 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:11.716961 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:11.716961 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:11.716961 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:11.716961 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:11.716961 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:11.716961 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:11.716961 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:11.716961 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:11.716961 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:11.716961 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:11.716961 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:11.716961 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:11.716961 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:11.716961 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:11.716961 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:03:11.718052 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:03:11.718052 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:11.718052 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:11.719340 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 237/462] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:03:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:11.720586 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ca71ad-62ca-40f6-8d4c-645c4d5d259f None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:11.720985 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ca71ad-62ca-40f6-8d4c-645c4d5d259f None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:11.721348 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ca71ad-62ca-40f6-8d4c-645c4d5d259f None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:11.722876 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 238/463] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:03:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:03:11.726620 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ca71ad-62ca-40f6-8d4c-645c4d5d259f None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:11.728735 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5efb8b5-d096-45b4-bf15-a531ef125251 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:11.765069 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b63f45f-0806-4ba8-a04e-558a8ffd847b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:11.765694 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b63f45f-0806-4ba8-a04e-558a8ffd847b None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:11.766015 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b63f45f-0806-4ba8-a04e-558a8ffd847b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:11.766279 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b63f45f-0806-4ba8-a04e-558a8ffd847b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:11.776486 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b63f45f-0806-4ba8-a04e-558a8ffd847b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:11.778031 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b63f45f-0806-4ba8-a04e-558a8ffd847b None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:11.793904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-4b63f45f-0806-4ba8-a04e-558a8ffd847b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:11.793904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:11.793904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:11.793904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:11.793904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:11.793904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:11.793904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:11.793904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:11.793904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:11.793904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:11.793904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:11.793904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:11.793904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:03:11.793904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:03:11.793904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:03:11.793904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:03:11.796440 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:11.796440 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:11.796440 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:11.796440 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:11.796440 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:11.796440 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:11.796440 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:11.796440 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:11.796440 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:11.796440 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:11.796440 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:11.796440 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:11.796440 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:11.796440 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:11.796440 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:11.796440 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:11.796440 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:03:11.798534 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:03:11.798534 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:11.798534 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:11.798534 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 226/464] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:03:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:03:11.798534 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ebcb56-8753-4c5a-9a5f-895bb48442f7 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:11.798534 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ebcb56-8753-4c5a-9a5f-895bb48442f7 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:11.798534 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ebcb56-8753-4c5a-9a5f-895bb48442f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:11.814630 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-58ebcb56-8753-4c5a-9a5f-895bb48442f7 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:03:11.815781 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-58ebcb56-8753-4c5a-9a5f-895bb48442f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:03:11.850691 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ca71ad-62ca-40f6-8d4c-645c4d5d259f None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:11.886495 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 227/465] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:03:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:03:11.887096 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ebcb56-8753-4c5a-9a5f-895bb48442f7 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:11.910326 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d83232-4be0-4894-aa3c-315ab8a59160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:11.910972 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d83232-4be0-4894-aa3c-315ab8a59160 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:11.911278 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d83232-4be0-4894-aa3c-315ab8a59160 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:11.911547 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d83232-4be0-4894-aa3c-315ab8a59160 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:11.912200 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d83232-4be0-4894-aa3c-315ab8a59160 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:11.912497 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d83232-4be0-4894-aa3c-315ab8a59160 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:11.913919 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d83232-4be0-4894-aa3c-315ab8a59160 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:11.931984 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 239/466] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:03:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:03:11.932929 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d83232-4be0-4894-aa3c-315ab8a59160 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:11.956653 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:11.980077 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:12.023544 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ebcb56-8753-4c5a-9a5f-895bb48442f7 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:12.042086 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d83232-4be0-4894-aa3c-315ab8a59160 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:12.073841 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1a3249-56da-40bb-bbe7-82e06e1cb02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:12.074896 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1a3249-56da-40bb-bbe7-82e06e1cb02d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:12.075393 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1a3249-56da-40bb-bbe7-82e06e1cb02d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:12.075885 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1a3249-56da-40bb-bbe7-82e06e1cb02d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:12.077381 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef99401-954e-4957-a9b3-ee385d87695b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:12.078182 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef99401-954e-4957-a9b3-ee385d87695b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:12.078536 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef99401-954e-4957-a9b3-ee385d87695b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:12.078901 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef99401-954e-4957-a9b3-ee385d87695b None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:12.079770 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef99401-954e-4957-a9b3-ee385d87695b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:12.080056 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef99401-954e-4957-a9b3-ee385d87695b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:12.081918 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef99401-954e-4957-a9b3-ee385d87695b None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:12.089377 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1a3249-56da-40bb-bbe7-82e06e1cb02d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:12.090842 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1a3249-56da-40bb-bbe7-82e06e1cb02d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:12.095842 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 240/467] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:03:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:12.096929 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef99401-954e-4957-a9b3-ee385d87695b None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:12.111214 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-4a1a3249-56da-40bb-bbe7-82e06e1cb02d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:12.111214 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:12.111214 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:12.111214 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:12.111214 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:12.111214 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:12.111214 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:12.111214 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:12.111214 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:12.111214 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:12.111214 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:12.111214 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:12.111214 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:03:12.111214 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:03:12.111214 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:03:12.111214 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:03:12.112128 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:12.112128 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:12.112128 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:12.112128 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:12.112128 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:12.112128 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:12.112128 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:12.112128 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:12.112128 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:12.112128 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:12.112128 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:12.112128 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:12.112128 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:12.112128 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:12.112128 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:12.112128 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:12.112128 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:03:12.113009 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:03:12.113009 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:12.113009 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:12.113009 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 228/468] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:03:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:12.113228 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1a3249-56da-40bb-bbe7-82e06e1cb02d None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:12.210684 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:12.214711 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef99401-954e-4957-a9b3-ee385d87695b None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:12.218501 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1a3249-56da-40bb-bbe7-82e06e1cb02d None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:12.228077 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:12.254919 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254d94b0-c038-4adb-9be4-e8ab8741d34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:12.255804 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254d94b0-c038-4adb-9be4-e8ab8741d34e None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:12.256042 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716a50fe-aee1-4d1a-858a-8bddf7b809ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:12.256157 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254d94b0-c038-4adb-9be4-e8ab8741d34e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:12.256321 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254d94b0-c038-4adb-9be4-e8ab8741d34e None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:12.256648 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716a50fe-aee1-4d1a-858a-8bddf7b809ae None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:12.256994 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254d94b0-c038-4adb-9be4-e8ab8741d34e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:12.257086 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716a50fe-aee1-4d1a-858a-8bddf7b809ae None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:12.257299 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716a50fe-aee1-4d1a-858a-8bddf7b809ae None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:12.257933 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716a50fe-aee1-4d1a-858a-8bddf7b809ae None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:12.258425 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254d94b0-c038-4adb-9be4-e8ab8741d34e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:12.259388 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716a50fe-aee1-4d1a-858a-8bddf7b809ae None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:12.284361 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 241/469] 10.222.0.43 () {60 vars in 1132 bytes} [Tue Sep 6 16:03:12 2022] POST /identity/v3/users => generated 248 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:03:12.284361 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254d94b0-c038-4adb-9be4-e8ab8741d34e None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:12.298305 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 229/470] 10.222.0.43 () {60 vars in 1132 bytes} [Tue Sep 6 16:03:12 2022] POST /identity/v3/users => generated 246 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:03:12.299207 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716a50fe-aee1-4d1a-858a-8bddf7b809ae None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:12.426054 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254d94b0-c038-4adb-9be4-e8ab8741d34e None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:12.482985 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a2f4ef-a13b-4604-8a72-004dcc36ddda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:12.483543 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a2f4ef-a13b-4604-8a72-004dcc36ddda None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:12.483818 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a2f4ef-a13b-4604-8a72-004dcc36ddda None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:12.484086 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a2f4ef-a13b-4604-8a72-004dcc36ddda None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:12.484775 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a2f4ef-a13b-4604-8a72-004dcc36ddda None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:12.485210 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a2f4ef-a13b-4604-8a72-004dcc36ddda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:12.486639 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a2f4ef-a13b-4604-8a72-004dcc36ddda None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:12.499836 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 242/471] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:03:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:12.501132 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080a2565-9840-4382-8305-87dbbfe10964 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:12.501419 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080a2565-9840-4382-8305-87dbbfe10964 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:12.501689 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716a50fe-aee1-4d1a-858a-8bddf7b809ae None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:12.501792 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080a2565-9840-4382-8305-87dbbfe10964 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:12.503181 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 243/472] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:03:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:03:12.505436 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080a2565-9840-4382-8305-87dbbfe10964 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:12.531219 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:12.553744 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a39401-b0c0-464d-ab4c-342981eb2cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:12.554669 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a39401-b0c0-464d-ab4c-342981eb2cfa None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:12.554946 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a39401-b0c0-464d-ab4c-342981eb2cfa None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:12.555211 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a39401-b0c0-464d-ab4c-342981eb2cfa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:12.568592 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a39401-b0c0-464d-ab4c-342981eb2cfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:12.570344 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a39401-b0c0-464d-ab4c-342981eb2cfa None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:12.587869 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-d6a39401-b0c0-464d-ab4c-342981eb2cfa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:12.587869 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:12.587869 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:12.587869 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:12.587869 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:12.587869 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:12.587869 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:12.587869 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:12.587869 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:12.587869 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:12.587869 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:12.587869 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:12.587869 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:03:12.587869 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:03:12.587869 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:03:12.587869 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:03:12.588904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:12.588904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:12.588904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:12.588904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:12.588904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:12.588904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:12.588904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:12.588904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:12.588904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:12.588904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:12.588904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:12.588904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:12.588904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:12.588904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:12.588904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:12.588904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:12.588904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:03:12.589995 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:03:12.589995 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:12.589995 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:12.589995 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 230/473] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:03:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:12.590790 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e491d7a8-c4ee-4ff3-a8e0-45958d4214e2 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:12.590790 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e491d7a8-c4ee-4ff3-a8e0-45958d4214e2 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:12.590790 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e491d7a8-c4ee-4ff3-a8e0-45958d4214e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:12.616522 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-e491d7a8-c4ee-4ff3-a8e0-45958d4214e2 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:03:12.618243 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e491d7a8-c4ee-4ff3-a8e0-45958d4214e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:03:12.629996 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:12.655665 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080a2565-9840-4382-8305-87dbbfe10964 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:12.697481 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 231/474] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:03:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:03:12.698349 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e491d7a8-c4ee-4ff3-a8e0-45958d4214e2 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:12.706028 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc886ab-2fc5-469c-9093-2e4002c1a0d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:12.706028 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc886ab-2fc5-469c-9093-2e4002c1a0d8 None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:12.706266 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc886ab-2fc5-469c-9093-2e4002c1a0d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:12.706336 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc886ab-2fc5-469c-9093-2e4002c1a0d8 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:12.707011 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc886ab-2fc5-469c-9093-2e4002c1a0d8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:12.709058 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc886ab-2fc5-469c-9093-2e4002c1a0d8 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:12.745206 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 244/475] 10.222.0.43 () {60 vars in 1132 bytes} [Tue Sep 6 16:03:12 2022] POST /identity/v3/users => generated 248 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:03:12.878548 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e491d7a8-c4ee-4ff3-a8e0-45958d4214e2 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:12.916865 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9c0c82-fd7b-46bb-b5cd-2bdd6b6d58b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:12.917489 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c0c82-fd7b-46bb-b5cd-2bdd6b6d58b2 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:12.917756 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c0c82-fd7b-46bb-b5cd-2bdd6b6d58b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:12.918073 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c0c82-fd7b-46bb-b5cd-2bdd6b6d58b2 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:12.918728 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9c0c82-fd7b-46bb-b5cd-2bdd6b6d58b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:12.919031 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9c0c82-fd7b-46bb-b5cd-2bdd6b6d58b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:12.920596 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9c0c82-fd7b-46bb-b5cd-2bdd6b6d58b2 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:12.932808 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 232/476] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:03:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:12.938788 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc886ab-2fc5-469c-9093-2e4002c1a0d8 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:13.016936 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:13.021639 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9c0c82-fd7b-46bb-b5cd-2bdd6b6d58b2 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:13.033861 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:13.059236 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc886ab-2fc5-469c-9093-2e4002c1a0d8 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:13.101315 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baddd187-7e1d-49fa-ab8a-8d6af4e98bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:13.101988 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baddd187-7e1d-49fa-ab8a-8d6af4e98bec None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:13.102418 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baddd187-7e1d-49fa-ab8a-8d6af4e98bec None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:13.102691 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baddd187-7e1d-49fa-ab8a-8d6af4e98bec None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:13.103910 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baddd187-7e1d-49fa-ab8a-8d6af4e98bec None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:13.105582 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baddd187-7e1d-49fa-ab8a-8d6af4e98bec None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:13.141556 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 245/477] 10.222.0.43 () {60 vars in 1132 bytes} [Tue Sep 6 16:03:12 2022] POST /identity/v3/users => generated 249 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:03:13.160485 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9c0c82-fd7b-46bb-b5cd-2bdd6b6d58b2 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:13.214484 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6943de7-3276-42af-9d09-fe8b9e303060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:13.215541 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6943de7-3276-42af-9d09-fe8b9e303060 None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:13.216020 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6943de7-3276-42af-9d09-fe8b9e303060 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:13.216487 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6943de7-3276-42af-9d09-fe8b9e303060 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:13.217477 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6943de7-3276-42af-9d09-fe8b9e303060 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:13.219104 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6943de7-3276-42af-9d09-fe8b9e303060 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:13.239181 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.sql.core [None req-c6943de7-3276-42af-9d09-fe8b9e303060 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 06 16:03:13.239181 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 06 16:03:13.239181 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [parameters: {'id': '5173d32d0ab048838679d77825e4563e', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 06 16:03:13.239181 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81698) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 06 16:03:13.264676 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-c6943de7-3276-42af-9d09-fe8b9e303060 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 06 16:03:13.264676 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:13.264676 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 06 16:03:13.264676 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 06 16:03:13.264676 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 06 16:03:13.264676 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 06 16:03:13.264676 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 06 16:03:13.264676 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application result = self._query(query) Sep 06 16:03:13.264676 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 06 16:03:13.264676 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application conn.query(q) Sep 06 16:03:13.264676 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 06 16:03:13.264676 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 06 16:03:13.264676 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 06 16:03:13.264676 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application result.read() Sep 06 16:03:13.264676 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 06 16:03:13.265653 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 06 16:03:13.265653 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 06 16:03:13.265653 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 06 16:03:13.265653 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 06 16:03:13.265653 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 06 16:03:13.265653 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 06 16:03:13.265653 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 06 16:03:13.265653 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 06 16:03:13.265653 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:13.265653 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 06 16:03:13.265653 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:13.265653 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:13.265653 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 06 16:03:13.265653 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 06 16:03:13.265653 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 06 16:03:13.265653 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return ref.to_dict() Sep 06 16:03:13.265653 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 06 16:03:13.265653 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application next(self.gen) Sep 06 16:03:13.266585 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 06 16:03:13.266585 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application yield resource Sep 06 16:03:13.266585 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 06 16:03:13.266585 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application next(self.gen) Sep 06 16:03:13.266585 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 06 16:03:13.266585 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self.session.rollback() Sep 06 16:03:13.266585 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 06 16:03:13.266585 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self.force_reraise() Sep 06 16:03:13.266585 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 06 16:03:13.266585 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise self.value Sep 06 16:03:13.266585 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 06 16:03:13.266585 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 06 16:03:13.266585 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 06 16:03:13.266585 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application session.commit() Sep 06 16:03:13.266585 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 06 16:03:13.266585 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 06 16:03:13.266585 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 06 16:03:13.267470 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._prepare_impl() Sep 06 16:03:13.267470 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 06 16:03:13.267470 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self.session.flush() Sep 06 16:03:13.267470 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 06 16:03:13.267470 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._flush(objects) Sep 06 16:03:13.267470 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 06 16:03:13.267470 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 06 16:03:13.267470 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 06 16:03:13.267470 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application compat.raise_( Sep 06 16:03:13.267470 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 06 16:03:13.267470 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception Sep 06 16:03:13.267470 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 06 16:03:13.267470 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application flush_context.execute() Sep 06 16:03:13.267470 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 06 16:03:13.267470 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rec.execute(self) Sep 06 16:03:13.267470 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 06 16:03:13.267470 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 06 16:03:13.268356 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 06 16:03:13.268356 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 06 16:03:13.268356 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 06 16:03:13.268356 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 06 16:03:13.268356 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 06 16:03:13.268356 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 06 16:03:13.268356 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 06 16:03:13.268356 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 06 16:03:13.268356 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 06 16:03:13.268356 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 06 16:03:13.268356 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 06 16:03:13.268356 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 06 16:03:13.268356 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 06 16:03:13.268356 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 06 16:03:13.268356 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 06 16:03:13.268356 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception Sep 06 16:03:13.269218 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 06 16:03:13.269218 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 06 16:03:13.269218 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 06 16:03:13.269218 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 06 16:03:13.269218 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 06 16:03:13.269218 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application result = self._query(query) Sep 06 16:03:13.269218 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 06 16:03:13.269218 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application conn.query(q) Sep 06 16:03:13.269218 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 06 16:03:13.269218 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 06 16:03:13.269218 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 06 16:03:13.269218 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application result.read() Sep 06 16:03:13.269218 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 06 16:03:13.269218 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 06 16:03:13.269218 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 06 16:03:13.269218 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 06 16:03:13.270010 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 06 16:03:13.270010 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 06 16:03:13.270010 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 06 16:03:13.270010 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 06 16:03:13.270010 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 06 16:03:13.270010 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 06 16:03:13.270010 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [parameters: {'id': '5173d32d0ab048838679d77825e4563e', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 06 16:03:13.270010 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 06 16:03:13.270010 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:13.270010 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 16:03:13.270010 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:13.270010 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:13.270010 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:13.270010 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:13.270010 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:13.270010 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:13.270864 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:13.270864 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:13.270864 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:13.270864 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:13.270864 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:13.270864 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:13.270864 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 06 16:03:13.270864 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 06 16:03:13.270864 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:13.270864 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:13.270864 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 06 16:03:13.270864 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 06 16:03:13.270864 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 06 16:03:13.270864 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 06 16:03:13.270864 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 06 16:03:13.270864 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:13.271738 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 233/478] 10.222.0.43 () {60 vars in 1131 bytes} [Tue Sep 6 16:03:13 2022] POST /identity/v3/roles => generated 144 bytes in 245 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:13.273163 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baddd187-7e1d-49fa-ab8a-8d6af4e98bec None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:13.399651 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baddd187-7e1d-49fa-ab8a-8d6af4e98bec None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:13.424481 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:13.433134 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30269725-0f4d-41e6-8d3b-f82c93d58dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:13.433777 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30269725-0f4d-41e6-8d3b-f82c93d58dc3 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:13.434055 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30269725-0f4d-41e6-8d3b-f82c93d58dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:13.434322 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30269725-0f4d-41e6-8d3b-f82c93d58dc3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:13.445716 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30269725-0f4d-41e6-8d3b-f82c93d58dc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:03:13.450775 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-30269725-0f4d-41e6-8d3b-f82c93d58dc3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 16:03:13.450775 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:13.450775 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:13.450775 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:13.450775 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:13.450775 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:13.450775 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:13.450775 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:13.450775 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:13.450775 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:13.450775 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:13.450775 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:13.450775 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:03:13.450775 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:03:13.450775 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:03:13.450775 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:03:13.452808 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:03:13.452808 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:03:13.452808 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:03:13.452808 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:03:13.452808 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:13.452808 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:13.452808 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:13.452808 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:13.452808 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:13.452808 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:13.452808 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:13.452808 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:13.452808 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:13.452808 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:13.452808 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:13.452808 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:13.454720 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:13.454720 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:13.454720 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:13.454720 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:13.454720 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:03:13.454720 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:03:13.454720 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:03:13.454720 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:03:13.454720 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:03:13.454720 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:03:13.454720 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 16:03:13.454720 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:13.454720 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 246/479] 10.222.0.43 () {56 vars in 1114 bytes} [Tue Sep 6 16:03:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:13.462049 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6943de7-3276-42af-9d09-fe8b9e303060 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:13.537144 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:13.573039 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6943de7-3276-42af-9d09-fe8b9e303060 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:13.611086 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81013f8b-6b43-4eb9-9ba1-00bfa247aac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:13.611703 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81013f8b-6b43-4eb9-9ba1-00bfa247aac1 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:13.611978 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81013f8b-6b43-4eb9-9ba1-00bfa247aac1 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:13.612237 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81013f8b-6b43-4eb9-9ba1-00bfa247aac1 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:13.612937 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81013f8b-6b43-4eb9-9ba1-00bfa247aac1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:13.613230 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81013f8b-6b43-4eb9-9ba1-00bfa247aac1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:13.614927 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81013f8b-6b43-4eb9-9ba1-00bfa247aac1 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:13.623804 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 234/480] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 6 16:03:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:14.548930 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0ef66b-073c-47f5-ad7e-673e1c61954a None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:14.549624 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0ef66b-073c-47f5-ad7e-673e1c61954a None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:14.549624 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0ef66b-073c-47f5-ad7e-673e1c61954a None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:14.551016 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 247/481] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:03:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:03:14.556784 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee220c-41df-4cef-b702-ec17ea811d74 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:14.557268 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee220c-41df-4cef-b702-ec17ea811d74 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:14.557431 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee220c-41df-4cef-b702-ec17ea811d74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:14.577035 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-f8ee220c-41df-4cef-b702-ec17ea811d74 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:03:14.578229 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f8ee220c-41df-4cef-b702-ec17ea811d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:03:14.652870 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 235/482] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:03:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 16:03:14.883108 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76773d64-9135-496d-a7b4-454494fb0ed8 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:14.883464 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76773d64-9135-496d-a7b4-454494fb0ed8 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:14.883650 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76773d64-9135-496d-a7b4-454494fb0ed8 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:14.885209 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 248/483] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:03:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:03:14.902222 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167f683b-2b98-4676-815a-97e1deff9c9c None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:14.902515 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167f683b-2b98-4676-815a-97e1deff9c9c None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:14.902788 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167f683b-2b98-4676-815a-97e1deff9c9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:14.925687 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-167f683b-2b98-4676-815a-97e1deff9c9c None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:03:14.926919 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-167f683b-2b98-4676-815a-97e1deff9c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:03:14.959086 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76773d64-9135-496d-a7b4-454494fb0ed8 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:14.990007 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:14.998035 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 236/484] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:03:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:03:15.086578 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038e9867-fc15-427a-8d1f-ee1dbb9dd439 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:15.087038 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038e9867-fc15-427a-8d1f-ee1dbb9dd439 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:15.087540 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038e9867-fc15-427a-8d1f-ee1dbb9dd439 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:15.089339 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 237/485] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:03:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:03:15.094435 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f4962-5bba-4fd2-ad01-84f12cc10dcd None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:15.094719 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f4962-5bba-4fd2-ad01-84f12cc10dcd None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:15.094984 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f4962-5bba-4fd2-ad01-84f12cc10dcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:15.116387 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-224f4962-5bba-4fd2-ad01-84f12cc10dcd None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:03:15.117620 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-224f4962-5bba-4fd2-ad01-84f12cc10dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:03:15.174152 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76773d64-9135-496d-a7b4-454494fb0ed8 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:15.200909 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 238/486] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:03:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:03:15.231837 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bdc5a5-df6c-4287-ab78-01c8256ca847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:15.232479 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bdc5a5-df6c-4287-ab78-01c8256ca847 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:15.232781 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bdc5a5-df6c-4287-ab78-01c8256ca847 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:15.233041 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bdc5a5-df6c-4287-ab78-01c8256ca847 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:15.249268 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bdc5a5-df6c-4287-ab78-01c8256ca847 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:03:15.254291 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-86bdc5a5-df6c-4287-ab78-01c8256ca847 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:03:15.254291 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:15.254291 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:15.254291 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:15.254291 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:15.254291 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:15.254291 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:15.254291 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:15.254291 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:15.254291 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:15.254291 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:15.254291 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:15.254291 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:03:15.254291 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:03:15.254291 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:03:15.254291 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:03:15.255204 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:03:15.255204 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:03:15.255204 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:03:15.255204 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:03:15.255204 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:15.255204 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:15.255204 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:15.255204 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:15.255204 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:15.255204 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:15.255204 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:15.255204 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:15.255204 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:15.255204 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:15.255204 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:15.255204 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:15.256000 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:15.256000 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:15.256000 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:15.256000 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:15.256000 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:03:15.256000 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:03:15.256000 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:03:15.256000 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:03:15.256000 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:03:15.256000 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:03:15.256000 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:03:15.256000 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:15.256000 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 249/487] 10.222.0.43 () {56 vars in 1102 bytes} [Tue Sep 6 16:03:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:15.261313 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224f4962-5bba-4fd2-ad01-84f12cc10dcd None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:15.355368 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bdc5a5-df6c-4287-ab78-01c8256ca847 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:15.396596 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224f4962-5bba-4fd2-ad01-84f12cc10dcd None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:15.421340 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:15.429492 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:15.450008 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd00fe8-12f1-4e85-b86f-23ab47b294ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:15.450900 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd00fe8-12f1-4e85-b86f-23ab47b294ee None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:15.451384 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd00fe8-12f1-4e85-b86f-23ab47b294ee None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:15.451846 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd00fe8-12f1-4e85-b86f-23ab47b294ee None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:15.452737 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd00fe8-12f1-4e85-b86f-23ab47b294ee None admin] RBAC: Adding query filter params (name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:15.453267 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd00fe8-12f1-4e85-b86f-23ab47b294ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:15.454973 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd00fe8-12f1-4e85-b86f-23ab47b294ee None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:15.471317 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 239/488] 10.222.0.43 () {56 vars in 1103 bytes} [Tue Sep 6 16:03:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:03:15.476897 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd00fe8-12f1-4e85-b86f-23ab47b294ee None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:15.485927 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bdc5a5-df6c-4287-ab78-01c8256ca847 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:15.522056 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35a8ea5-ff57-465e-8189-e1c979b3e19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:15.522920 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35a8ea5-ff57-465e-8189-e1c979b3e19b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:15.523337 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35a8ea5-ff57-465e-8189-e1c979b3e19b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:15.523972 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35a8ea5-ff57-465e-8189-e1c979b3e19b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:15.536613 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35a8ea5-ff57-465e-8189-e1c979b3e19b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:03:15.543092 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-d35a8ea5-ff57-465e-8189-e1c979b3e19b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:03:15.543092 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:15.543092 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:15.543092 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:15.543092 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:15.543092 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:15.543092 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:15.543092 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:15.543092 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:15.543092 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:15.543092 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:15.543092 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:15.543092 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:03:15.543092 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:03:15.543092 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:03:15.543092 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:03:15.543972 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:03:15.543972 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:03:15.543972 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:03:15.543972 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:03:15.543972 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:15.543972 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:15.543972 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:15.543972 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:15.543972 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:15.543972 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:15.543972 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:15.543972 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:15.543972 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:15.543972 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:15.543972 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:15.543972 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:15.544791 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:15.544791 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:15.544791 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:15.544791 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:15.544791 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:03:15.544791 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:03:15.544791 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:03:15.544791 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:03:15.544791 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:03:15.544791 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:03:15.544791 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:03:15.544791 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:15.546948 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 250/489] 10.222.0.43 () {56 vars in 1102 bytes} [Tue Sep 6 16:03:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:03:15.554304 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35a8ea5-ff57-465e-8189-e1c979b3e19b None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:15.614171 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd00fe8-12f1-4e85-b86f-23ab47b294ee None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:15.648922 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b9a2e-bc1f-4d8e-9e21-f502246d8d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:15.649911 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b9a2e-bc1f-4d8e-9e21-f502246d8d9c None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:15.650386 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b9a2e-bc1f-4d8e-9e21-f502246d8d9c None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:15.650850 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b9a2e-bc1f-4d8e-9e21-f502246d8d9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:15.662901 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536b9a2e-bc1f-4d8e-9e21-f502246d8d9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:15.664846 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536b9a2e-bc1f-4d8e-9e21-f502246d8d9c None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:15.678678 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35a8ea5-ff57-465e-8189-e1c979b3e19b None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:15.681917 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-536b9a2e-bc1f-4d8e-9e21-f502246d8d9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:15.681917 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:15.681917 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:15.681917 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:15.681917 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:15.681917 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:15.681917 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:15.681917 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:15.681917 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:15.681917 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:15.681917 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:15.681917 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:15.681917 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:03:15.681917 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:03:15.681917 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:03:15.681917 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:03:15.682821 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:15.682821 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:15.682821 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:15.682821 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:15.682821 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:15.682821 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:15.682821 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:15.682821 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:15.682821 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:15.682821 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:15.682821 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:15.682821 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:15.682821 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:15.682821 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:15.682821 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:15.682821 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:15.682821 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:03:15.683675 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:03:15.683675 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:15.683675 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:15.686750 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 240/490] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:03:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:15.687549 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:15.690535 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b9a2e-bc1f-4d8e-9e21-f502246d8d9c None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:15.722959 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b952b82a-4ba0-41d8-9029-32ff00025a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:15.724004 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b952b82a-4ba0-41d8-9029-32ff00025a21 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:15.724584 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b952b82a-4ba0-41d8-9029-32ff00025a21 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:15.725084 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b952b82a-4ba0-41d8-9029-32ff00025a21 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:15.726048 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b952b82a-4ba0-41d8-9029-32ff00025a21 None admin] RBAC: Adding query filter params (name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:15.726654 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b952b82a-4ba0-41d8-9029-32ff00025a21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:15.728541 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b952b82a-4ba0-41d8-9029-32ff00025a21 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:15.731880 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:15.739556 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 251/491] 10.222.0.43 () {56 vars in 1103 bytes} [Tue Sep 6 16:03:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:03:15.740715 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b952b82a-4ba0-41d8-9029-32ff00025a21 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:15.800518 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b9a2e-bc1f-4d8e-9e21-f502246d8d9c None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:15.836229 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad18cbea-c3c0-46b4-9367-b67d6f54bff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:15.836895 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad18cbea-c3c0-46b4-9367-b67d6f54bff2 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:15.837178 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad18cbea-c3c0-46b4-9367-b67d6f54bff2 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:15.837445 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad18cbea-c3c0-46b4-9367-b67d6f54bff2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:15.843759 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b952b82a-4ba0-41d8-9029-32ff00025a21 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:15.847796 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad18cbea-c3c0-46b4-9367-b67d6f54bff2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:03:15.853106 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-ad18cbea-c3c0-46b4-9367-b67d6f54bff2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:03:15.853106 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:15.853106 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:15.853106 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:15.853106 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:15.853106 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:15.853106 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:15.853106 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:15.853106 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:15.853106 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:15.853106 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:15.853106 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:15.853106 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:03:15.853106 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:03:15.853106 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:03:15.853106 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:03:15.854152 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:03:15.854152 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:03:15.854152 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:03:15.854152 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:03:15.854152 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:15.854152 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:15.854152 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:15.854152 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:15.854152 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:15.854152 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:15.854152 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:15.854152 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:15.854152 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:15.854152 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:15.854152 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:15.854152 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:15.855095 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:15.855095 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:15.855095 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:15.855095 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:15.855095 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:03:15.855095 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:03:15.855095 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:03:15.855095 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:03:15.855095 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:03:15.855095 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:03:15.855095 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:03:15.855095 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:15.855095 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 241/492] 10.222.0.43 () {56 vars in 1102 bytes} [Tue Sep 6 16:03:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:15.855095 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad18cbea-c3c0-46b4-9367-b67d6f54bff2 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:15.882272 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad534a65-3d73-405c-a3f6-08c1ef7b9ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:15.882975 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad534a65-3d73-405c-a3f6-08c1ef7b9ad8 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:15.883319 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad534a65-3d73-405c-a3f6-08c1ef7b9ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:15.883591 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad534a65-3d73-405c-a3f6-08c1ef7b9ad8 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:15.884246 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad534a65-3d73-405c-a3f6-08c1ef7b9ad8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:15.884545 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad534a65-3d73-405c-a3f6-08c1ef7b9ad8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:15.885980 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad534a65-3d73-405c-a3f6-08c1ef7b9ad8 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:15.897832 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 252/493] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:03:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:15.899090 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a854ddb3-c6ce-487f-a3f5-bc5b1fca1562 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:15.899374 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a854ddb3-c6ce-487f-a3f5-bc5b1fca1562 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:15.899647 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a854ddb3-c6ce-487f-a3f5-bc5b1fca1562 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:15.901135 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 253/494] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:03:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:03:15.901827 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a854ddb3-c6ce-487f-a3f5-bc5b1fca1562 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:15.929869 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:15.963778 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:15.971985 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad18cbea-c3c0-46b4-9367-b67d6f54bff2 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:16.008481 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a854ddb3-c6ce-487f-a3f5-bc5b1fca1562 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:16.009594 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cabd17-558a-4c58-b1f5-95ddeb504f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:16.010221 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cabd17-558a-4c58-b1f5-95ddeb504f2d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:16.010491 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cabd17-558a-4c58-b1f5-95ddeb504f2d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:16.010755 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cabd17-558a-4c58-b1f5-95ddeb504f2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:16.022101 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cabd17-558a-4c58-b1f5-95ddeb504f2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:16.024129 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cabd17-558a-4c58-b1f5-95ddeb504f2d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:16.039200 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-85cabd17-558a-4c58-b1f5-95ddeb504f2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:16.039200 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:16.039200 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:16.039200 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:16.039200 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:16.039200 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:16.039200 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:16.039200 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:16.039200 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:16.039200 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:16.039200 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:16.039200 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:16.039200 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:03:16.039200 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:03:16.039200 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:03:16.039200 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:03:16.040081 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:16.040081 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:16.040081 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:16.040081 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:16.040081 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:16.040081 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:16.040081 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:16.040081 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:16.040081 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:16.040081 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:16.040081 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:16.040081 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:16.040081 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:16.040081 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:16.040081 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:16.040081 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:16.040081 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:03:16.040956 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:03:16.040956 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:16.040956 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:16.040956 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 242/495] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:03:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:16.041180 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cabd17-558a-4c58-b1f5-95ddeb504f2d None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:16.049316 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d917a66-a5cc-4aa1-bea1-0f2845ee811d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:16.049936 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d917a66-a5cc-4aa1-bea1-0f2845ee811d None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:16.050213 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d917a66-a5cc-4aa1-bea1-0f2845ee811d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:16.050485 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d917a66-a5cc-4aa1-bea1-0f2845ee811d None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:16.051152 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d917a66-a5cc-4aa1-bea1-0f2845ee811d None admin] RBAC: Adding query filter params (name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:16.051523 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d917a66-a5cc-4aa1-bea1-0f2845ee811d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:16.052933 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d917a66-a5cc-4aa1-bea1-0f2845ee811d None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:16.065776 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 254/496] 10.222.0.43 () {56 vars in 1103 bytes} [Tue Sep 6 16:03:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:16.065776 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a7caa4-9a37-40aa-8a51-8a8f02b503ee None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:16.065776 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a7caa4-9a37-40aa-8a51-8a8f02b503ee None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:16.065776 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a7caa4-9a37-40aa-8a51-8a8f02b503ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:16.081717 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-12a7caa4-9a37-40aa-8a51-8a8f02b503ee None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:03:16.083035 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-12a7caa4-9a37-40aa-8a51-8a8f02b503ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:03:16.149490 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cabd17-558a-4c58-b1f5-95ddeb504f2d None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:16.158448 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 255/497] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:03:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:03:16.159655 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a7caa4-9a37-40aa-8a51-8a8f02b503ee None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:16.182936 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1f7306-cdc4-4021-ad0e-af3f234a83fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:16.183570 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1f7306-cdc4-4021-ad0e-af3f234a83fa None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:16.183844 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1f7306-cdc4-4021-ad0e-af3f234a83fa None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:16.184215 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1f7306-cdc4-4021-ad0e-af3f234a83fa None admin] PATH_INFO: `/v3/users/glance` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:16.192480 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:16.193769 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1f7306-cdc4-4021-ad0e-af3f234a83fa None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:16.195474 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1f7306-cdc4-4021-ad0e-af3f234a83fa None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:16.209325 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-af1f7306-cdc4-4021-ad0e-af3f234a83fa None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 06 16:03:16.209325 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:16.209325 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:16.209325 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:16.209325 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:16.209325 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:16.209325 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:16.209325 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:16.209325 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:16.209325 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:16.209325 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:16.209325 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:16.209325 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:03:16.209325 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:03:16.209325 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:03:16.209325 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:03:16.210436 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:16.210436 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:16.210436 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:03:16.210436 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:03:16.210436 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:03:16.210436 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:03:16.210436 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:16.210436 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:16.210436 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:16.210436 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:16.210436 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:16.210436 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:16.210436 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:16.210436 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:16.210436 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:16.210436 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:16.210436 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:16.210436 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:16.211537 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:16.211537 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:16.211537 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:03:16.211537 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:03:16.211537 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:03:16.211537 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:03:16.211537 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:03:16.211537 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:03:16.211537 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 06 16:03:16.211537 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:16.213349 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 243/498] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 6 16:03:16 2022] GET /identity/v3/users/glance => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:16.218425 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1f7306-cdc4-4021-ad0e-af3f234a83fa None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:16.230292 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:16.293036 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a7caa4-9a37-40aa-8a51-8a8f02b503ee None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:16.319819 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1f7306-cdc4-4021-ad0e-af3f234a83fa None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:16.327245 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e34f8b-196c-4b1e-be0c-8a711782f6ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:16.327855 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e34f8b-196c-4b1e-be0c-8a711782f6ae None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:16.328138 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e34f8b-196c-4b1e-be0c-8a711782f6ae None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:16.328405 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e34f8b-196c-4b1e-be0c-8a711782f6ae None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:16.329082 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e34f8b-196c-4b1e-be0c-8a711782f6ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:16.329379 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e34f8b-196c-4b1e-be0c-8a711782f6ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:16.331146 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e34f8b-196c-4b1e-be0c-8a711782f6ae None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:16.343059 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 256/499] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:03:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:16.343979 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388d5482-60af-4b4e-adc8-514da60e32f6 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:16.344265 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388d5482-60af-4b4e-adc8-514da60e32f6 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:16.344539 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388d5482-60af-4b4e-adc8-514da60e32f6 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:16.345766 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 257/500] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:03:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:03:16.349022 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388d5482-60af-4b4e-adc8-514da60e32f6 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:16.361480 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab7391b-4be1-43c3-90db-6a7af0dac707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:16.362114 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab7391b-4be1-43c3-90db-6a7af0dac707 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:16.362388 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab7391b-4be1-43c3-90db-6a7af0dac707 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:16.362656 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab7391b-4be1-43c3-90db-6a7af0dac707 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:16.373424 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab7391b-4be1-43c3-90db-6a7af0dac707 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:16.374905 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab7391b-4be1-43c3-90db-6a7af0dac707 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:16.389220 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-6ab7391b-4be1-43c3-90db-6a7af0dac707 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:16.389220 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:16.389220 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:16.389220 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:16.389220 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:16.389220 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:16.389220 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:16.389220 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:16.389220 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:16.389220 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:16.389220 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:16.389220 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:16.389220 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:03:16.389220 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:03:16.389220 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:03:16.389220 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:03:16.390111 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:16.390111 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:16.390111 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:16.390111 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:16.390111 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:16.390111 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:16.390111 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:16.390111 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:16.390111 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:16.390111 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:16.390111 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:16.390111 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:16.390111 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:16.390111 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:16.390111 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:16.390111 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:16.390111 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:03:16.390962 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:03:16.390962 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:16.390962 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:16.390962 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 244/501] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:03:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:16.391186 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab7391b-4be1-43c3-90db-6a7af0dac707 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:16.438974 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:16.454095 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:16.459875 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388d5482-60af-4b4e-adc8-514da60e32f6 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:16.492398 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2dcc6f-4d4d-42b6-a22e-d6f2e85dbe50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:16.492949 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2dcc6f-4d4d-42b6-a22e-d6f2e85dbe50 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:16.493207 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2dcc6f-4d4d-42b6-a22e-d6f2e85dbe50 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:16.493471 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2dcc6f-4d4d-42b6-a22e-d6f2e85dbe50 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:16.494184 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2dcc6f-4d4d-42b6-a22e-d6f2e85dbe50 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:16.494288 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab7391b-4be1-43c3-90db-6a7af0dac707 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:16.494482 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2dcc6f-4d4d-42b6-a22e-d6f2e85dbe50 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:16.495913 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2dcc6f-4d4d-42b6-a22e-d6f2e85dbe50 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:16.526654 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444eba48-7bdc-4a18-9a5d-06a5903e3aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:16.527253 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444eba48-7bdc-4a18-9a5d-06a5903e3aa3 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:16.527527 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444eba48-7bdc-4a18-9a5d-06a5903e3aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:16.527791 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444eba48-7bdc-4a18-9a5d-06a5903e3aa3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:16.530636 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 258/502] 10.222.0.43 () {56 vars in 1137 bytes} [Tue Sep 6 16:03:16 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:16.531818 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917429a-1903-472c-be11-dee656ed828d None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:16.532096 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917429a-1903-472c-be11-dee656ed828d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:16.532362 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917429a-1903-472c-be11-dee656ed828d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:16.536295 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444eba48-7bdc-4a18-9a5d-06a5903e3aa3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:16.537789 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444eba48-7bdc-4a18-9a5d-06a5903e3aa3 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:16.548845 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-444eba48-7bdc-4a18-9a5d-06a5903e3aa3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 16:03:16.548845 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:16.548845 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:16.548845 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:16.548845 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:16.548845 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:16.548845 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:16.548845 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:16.548845 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:16.548845 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:16.548845 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:16.548845 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:16.548845 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:03:16.548845 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:03:16.548845 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:03:16.548845 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:03:16.549831 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:16.549831 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:16.549831 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:03:16.549831 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:03:16.549831 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:03:16.549831 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:03:16.549831 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:16.549831 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:16.549831 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:16.549831 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:16.549831 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:16.549831 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:16.549831 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:16.549831 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:16.549831 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:16.549831 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:16.549831 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:16.549831 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:16.550720 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:16.550720 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:16.550720 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:03:16.550720 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:03:16.550720 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:03:16.550720 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:03:16.550720 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:03:16.550720 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:03:16.550720 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 16:03:16.550720 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:16.551682 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-4917429a-1903-472c-be11-dee656ed828d None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:03:16.551766 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 245/503] 10.222.0.43 () {56 vars in 1093 bytes} [Tue Sep 6 16:03:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:16.551766 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444eba48-7bdc-4a18-9a5d-06a5903e3aa3 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:16.551906 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-4917429a-1903-472c-be11-dee656ed828d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:03:16.615942 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 259/504] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:03:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:03:16.616785 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4917429a-1903-472c-be11-dee656ed828d None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:16.664687 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444eba48-7bdc-4a18-9a5d-06a5903e3aa3 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:16.684849 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:16.686421 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:16.701241 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb98314-5dae-4ae8-8b4b-fe4e950ae6ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:16.701845 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb98314-5dae-4ae8-8b4b-fe4e950ae6ca None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:16.702122 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb98314-5dae-4ae8-8b4b-fe4e950ae6ca None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:16.702394 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb98314-5dae-4ae8-8b4b-fe4e950ae6ca None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:16.703040 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb98314-5dae-4ae8-8b4b-fe4e950ae6ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:16.703346 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb98314-5dae-4ae8-8b4b-fe4e950ae6ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:16.704736 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb98314-5dae-4ae8-8b4b-fe4e950ae6ca None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:16.717751 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 246/505] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:03:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:16.718594 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb98314-5dae-4ae8-8b4b-fe4e950ae6ca None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:16.735126 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4917429a-1903-472c-be11-dee656ed828d None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:16.776288 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6268c2a-e3b8-489b-8d7c-60c5ec2a0e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:16.777254 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6268c2a-e3b8-489b-8d7c-60c5ec2a0e2b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:16.777709 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6268c2a-e3b8-489b-8d7c-60c5ec2a0e2b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:16.777974 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6268c2a-e3b8-489b-8d7c-60c5ec2a0e2b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:16.788772 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6268c2a-e3b8-489b-8d7c-60c5ec2a0e2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:03:16.793898 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-f6268c2a-e3b8-489b-8d7c-60c5ec2a0e2b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:03:16.793898 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:16.793898 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:16.793898 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:16.793898 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:16.793898 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:16.793898 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:16.793898 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:16.793898 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:16.793898 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:16.793898 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:16.793898 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:16.793898 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:03:16.793898 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:03:16.793898 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:03:16.793898 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:03:16.794784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:03:16.794784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:03:16.794784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:03:16.794784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:03:16.794784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:16.794784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:16.794784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:16.794784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:16.794784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:16.794784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:16.794784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:16.794784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:16.794784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:16.794784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:16.794784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:16.794784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:16.795587 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:16.795587 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:16.795587 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:16.795587 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:16.795587 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:03:16.795587 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:03:16.795587 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:03:16.795587 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:03:16.795587 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:03:16.795587 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:03:16.795587 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:03:16.795587 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:16.795587 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 260/506] 10.222.0.43 () {56 vars in 1102 bytes} [Tue Sep 6 16:03:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:16.796235 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6268c2a-e3b8-489b-8d7c-60c5ec2a0e2b None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:16.826736 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb98314-5dae-4ae8-8b4b-fe4e950ae6ca None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:16.862899 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd98b09-98b3-4a29-a1b5-1bd4c511a4b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:16.863478 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd98b09-98b3-4a29-a1b5-1bd4c511a4b9 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:16.863751 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd98b09-98b3-4a29-a1b5-1bd4c511a4b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:16.864068 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd98b09-98b3-4a29-a1b5-1bd4c511a4b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:16.893726 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6268c2a-e3b8-489b-8d7c-60c5ec2a0e2b None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:16.912532 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:16.923207 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:16.931008 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92803eb-a258-44fe-9454-ab700f4129bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:16.931614 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92803eb-a258-44fe-9454-ab700f4129bd None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:16.931886 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92803eb-a258-44fe-9454-ab700f4129bd None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:16.932154 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92803eb-a258-44fe-9454-ab700f4129bd None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:16.932589 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd98b09-98b3-4a29-a1b5-1bd4c511a4b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:16.932920 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92803eb-a258-44fe-9454-ab700f4129bd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:16.933218 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92803eb-a258-44fe-9454-ab700f4129bd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:16.934190 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd98b09-98b3-4a29-a1b5-1bd4c511a4b9 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:16.934635 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92803eb-a258-44fe-9454-ab700f4129bd None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:16.952227 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 261/507] 10.222.0.43 () {56 vars in 1133 bytes} [Tue Sep 6 16:03:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:16.953053 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92803eb-a258-44fe-9454-ab700f4129bd None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:17.030552 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 247/508] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 6 16:03:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 16:03:17.031310 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd98b09-98b3-4a29-a1b5-1bd4c511a4b9 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:17.052336 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92803eb-a258-44fe-9454-ab700f4129bd None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:17.086271 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f17d525-c954-4b7a-b806-67b5d8b4405b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:17.086898 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f17d525-c954-4b7a-b806-67b5d8b4405b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:17.087165 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f17d525-c954-4b7a-b806-67b5d8b4405b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:17.087436 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f17d525-c954-4b7a-b806-67b5d8b4405b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:17.094828 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f17d525-c954-4b7a-b806-67b5d8b4405b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:17.096286 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f17d525-c954-4b7a-b806-67b5d8b4405b None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:17.108996 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-3f17d525-c954-4b7a-b806-67b5d8b4405b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 06 16:03:17.108996 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:17.108996 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:17.108996 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:17.108996 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:17.108996 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:17.108996 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:17.108996 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:17.108996 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:17.108996 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:17.108996 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:17.108996 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:17.108996 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:03:17.108996 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:03:17.108996 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:03:17.108996 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:03:17.109915 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:17.109915 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:17.109915 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:03:17.109915 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:03:17.109915 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:03:17.109915 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:03:17.109915 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:17.109915 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:17.109915 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:17.109915 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:17.109915 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:17.109915 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:17.109915 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:17.109915 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:17.109915 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:17.109915 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:17.109915 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:17.109915 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:17.110790 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:17.110790 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:17.110790 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:03:17.110790 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:03:17.110790 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:03:17.110790 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:03:17.110790 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:03:17.110790 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:03:17.110790 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 06 16:03:17.110790 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:17.110790 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 262/509] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 6 16:03:16 2022] GET /identity/v3/users/cinder => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:17.111347 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f17d525-c954-4b7a-b806-67b5d8b4405b None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:17.111888 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:17.133393 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd98b09-98b3-4a29-a1b5-1bd4c511a4b9 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:17.159219 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:17.166797 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5b2db5-8df2-402c-a355-8c295caf8a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:17.167391 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5b2db5-8df2-402c-a355-8c295caf8a5f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:17.167664 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5b2db5-8df2-402c-a355-8c295caf8a5f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:17.167932 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5b2db5-8df2-402c-a355-8c295caf8a5f None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:17.168592 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5b2db5-8df2-402c-a355-8c295caf8a5f None admin] RBAC: Adding query filter params (name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:17.168918 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5b2db5-8df2-402c-a355-8c295caf8a5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:17.170305 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5b2db5-8df2-402c-a355-8c295caf8a5f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:17.178982 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 248/510] 10.222.0.43 () {56 vars in 1103 bytes} [Tue Sep 6 16:03:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:17.181625 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5b2db5-8df2-402c-a355-8c295caf8a5f None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:17.214864 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f17d525-c954-4b7a-b806-67b5d8b4405b None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:17.253301 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b64e099-8f0a-4efa-8c8a-1a9f693e6d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:17.253929 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b64e099-8f0a-4efa-8c8a-1a9f693e6d74 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:17.254206 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b64e099-8f0a-4efa-8c8a-1a9f693e6d74 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:17.254595 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b64e099-8f0a-4efa-8c8a-1a9f693e6d74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:17.265107 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b64e099-8f0a-4efa-8c8a-1a9f693e6d74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:17.266556 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b64e099-8f0a-4efa-8c8a-1a9f693e6d74 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:17.281273 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5b2db5-8df2-402c-a355-8c295caf8a5f None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:17.281408 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-1b64e099-8f0a-4efa-8c8a-1a9f693e6d74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:17.281408 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:17.281408 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:17.281408 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:17.281408 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:17.281408 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:17.281408 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:17.281408 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:17.281408 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:17.281408 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:17.281408 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:17.281408 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:17.281408 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:03:17.281408 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:03:17.281408 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:03:17.281408 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:03:17.282422 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:17.282422 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:17.282422 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:17.282422 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:17.282422 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:17.282422 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:17.282422 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:17.282422 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:17.282422 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:17.282422 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:17.282422 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:17.282422 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:17.282422 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:17.282422 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:17.282422 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:17.282422 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:17.282422 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:03:17.283483 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:03:17.283483 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:17.283483 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:17.287517 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 263/511] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:03:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:17.288349 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b64e099-8f0a-4efa-8c8a-1a9f693e6d74 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:17.315807 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492fa081-c9fd-41bf-a02b-377e676225c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:17.316417 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492fa081-c9fd-41bf-a02b-377e676225c4 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:17.316695 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492fa081-c9fd-41bf-a02b-377e676225c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:17.317083 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492fa081-c9fd-41bf-a02b-377e676225c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:17.328664 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:17.383018 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:17.384125 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492fa081-c9fd-41bf-a02b-377e676225c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:17.385838 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492fa081-c9fd-41bf-a02b-377e676225c4 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:17.386962 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b64e099-8f0a-4efa-8c8a-1a9f693e6d74 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:17.419428 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d388a119-507c-4082-b297-4f66944c7889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:17.420046 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d388a119-507c-4082-b297-4f66944c7889 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:17.420323 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d388a119-507c-4082-b297-4f66944c7889 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:17.420592 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d388a119-507c-4082-b297-4f66944c7889 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:17.430736 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d388a119-507c-4082-b297-4f66944c7889 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:17.432192 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d388a119-507c-4082-b297-4f66944c7889 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:17.446014 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-d388a119-507c-4082-b297-4f66944c7889 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:17.446014 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:17.446014 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:17.446014 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:17.446014 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:17.446014 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:17.446014 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:17.446014 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:17.446014 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:17.446014 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:17.446014 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:17.446014 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:17.446014 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:03:17.446014 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:03:17.446014 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:03:17.446014 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:03:17.446949 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:17.446949 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:17.446949 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:17.446949 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:17.446949 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:17.446949 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:17.446949 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:17.446949 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:17.446949 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:17.446949 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:17.446949 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:17.446949 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:17.446949 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:17.446949 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:17.446949 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:17.446949 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:17.446949 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:03:17.447796 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:03:17.447796 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:17.447796 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:17.447796 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 264/512] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:03:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:17.448018 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d388a119-507c-4082-b297-4f66944c7889 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:17.484212 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 249/513] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 6 16:03:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 16:03:17.485071 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492fa081-c9fd-41bf-a02b-377e676225c4 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:17.526413 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:17.548358 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d388a119-507c-4082-b297-4f66944c7889 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:17.583111 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492fa081-c9fd-41bf-a02b-377e676225c4 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:17.587903 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2596fec3-7de7-4ab0-9711-cdf59eb7524f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:17.588512 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2596fec3-7de7-4ab0-9711-cdf59eb7524f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:17.588811 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2596fec3-7de7-4ab0-9711-cdf59eb7524f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:17.589083 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2596fec3-7de7-4ab0-9711-cdf59eb7524f None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:17.589808 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2596fec3-7de7-4ab0-9711-cdf59eb7524f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:17.590407 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2596fec3-7de7-4ab0-9711-cdf59eb7524f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:17.591554 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2596fec3-7de7-4ab0-9711-cdf59eb7524f None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:17.607911 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 265/514] 10.222.0.43 () {56 vars in 1137 bytes} [Tue Sep 6 16:03:17 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:17.608687 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2596fec3-7de7-4ab0-9711-cdf59eb7524f None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:17.616606 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef2e037-38be-4d74-927f-c23cd7c56f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:17.617265 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef2e037-38be-4d74-927f-c23cd7c56f38 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:17.617537 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef2e037-38be-4d74-927f-c23cd7c56f38 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:17.617803 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef2e037-38be-4d74-927f-c23cd7c56f38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:17.620339 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:17.628079 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef2e037-38be-4d74-927f-c23cd7c56f38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:17.629518 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef2e037-38be-4d74-927f-c23cd7c56f38 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:17.643672 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-7ef2e037-38be-4d74-927f-c23cd7c56f38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:17.643672 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:17.643672 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:17.643672 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:17.643672 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:17.643672 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:17.643672 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:17.643672 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:17.643672 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:17.643672 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:17.643672 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:17.643672 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:17.643672 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:03:17.643672 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:03:17.643672 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:03:17.643672 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:03:17.644578 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:17.644578 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:17.644578 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:17.644578 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:17.644578 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:17.644578 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:17.644578 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:17.644578 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:17.644578 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:17.644578 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:17.644578 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:17.644578 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:17.644578 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:17.644578 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:17.644578 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:17.644578 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:17.644578 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:03:17.645528 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:03:17.645528 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:17.645528 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:17.645528 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 250/515] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:03:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:17.645854 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef2e037-38be-4d74-927f-c23cd7c56f38 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:17.710924 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2596fec3-7de7-4ab0-9711-cdf59eb7524f None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:17.738210 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:17.743985 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ab5491-e7cf-4e8a-a31e-d6f706fe0973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:17.744595 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab5491-e7cf-4e8a-a31e-d6f706fe0973 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:17.744912 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab5491-e7cf-4e8a-a31e-d6f706fe0973 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:17.745187 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab5491-e7cf-4e8a-a31e-d6f706fe0973 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:17.745812 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef2e037-38be-4d74-927f-c23cd7c56f38 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:17.745921 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ab5491-e7cf-4e8a-a31e-d6f706fe0973 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:17.746118 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ab5491-e7cf-4e8a-a31e-d6f706fe0973 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:17.747498 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ab5491-e7cf-4e8a-a31e-d6f706fe0973 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:17.759033 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 266/516] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:03:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:17.759866 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ab5491-e7cf-4e8a-a31e-d6f706fe0973 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:17.785927 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccc138e-6e67-4007-9674-bd20a83e8324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:17.786538 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc138e-6e67-4007-9674-bd20a83e8324 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:17.786814 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc138e-6e67-4007-9674-bd20a83e8324 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:17.787083 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc138e-6e67-4007-9674-bd20a83e8324 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:17.787729 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccc138e-6e67-4007-9674-bd20a83e8324 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:17.788017 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccc138e-6e67-4007-9674-bd20a83e8324 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:17.789440 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccc138e-6e67-4007-9674-bd20a83e8324 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:17.800929 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 251/517] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:03:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:17.801650 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccc138e-6e67-4007-9674-bd20a83e8324 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:17.825853 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:17.860636 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ab5491-e7cf-4e8a-a31e-d6f706fe0973 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:17.893572 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab53508-3e06-403b-b1d8-51fbfeb5c00d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:17.894194 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab53508-3e06-403b-b1d8-51fbfeb5c00d None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:17.894472 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab53508-3e06-403b-b1d8-51fbfeb5c00d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:17.894736 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab53508-3e06-403b-b1d8-51fbfeb5c00d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:17.901951 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccc138e-6e67-4007-9674-bd20a83e8324 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:17.905822 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab53508-3e06-403b-b1d8-51fbfeb5c00d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:17.907266 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab53508-3e06-403b-b1d8-51fbfeb5c00d None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:17.921702 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-0ab53508-3e06-403b-b1d8-51fbfeb5c00d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:17.921702 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:17.921702 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:17.921702 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:17.921702 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:17.921702 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:17.921702 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:17.921702 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:17.921702 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:17.921702 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:17.921702 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:17.921702 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:17.921702 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:03:17.921702 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:03:17.921702 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:03:17.921702 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:03:17.922781 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:17.922781 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:17.922781 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:17.922781 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:17.922781 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:17.922781 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:17.922781 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:17.922781 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:17.922781 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:17.922781 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:17.922781 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:17.922781 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:17.922781 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:17.922781 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:17.922781 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:17.922781 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:17.922781 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:03:17.923823 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:03:17.923823 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:17.923823 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:17.923823 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 267/518] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:03:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:17.924069 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab53508-3e06-403b-b1d8-51fbfeb5c00d None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:17.937611 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c61957e-6789-48fb-8b6d-e4b328d5f1ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:17.938218 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c61957e-6789-48fb-8b6d-e4b328d5f1ed None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:17.938496 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c61957e-6789-48fb-8b6d-e4b328d5f1ed None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:17.938761 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c61957e-6789-48fb-8b6d-e4b328d5f1ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:17.962651 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:18.005670 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c61957e-6789-48fb-8b6d-e4b328d5f1ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:18.007171 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c61957e-6789-48fb-8b6d-e4b328d5f1ed None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:18.023238 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab53508-3e06-403b-b1d8-51fbfeb5c00d None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:18.042070 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:18.055748 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3244f6-5745-4daf-8348-5dae00b7d85b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:18.056279 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3244f6-5745-4daf-8348-5dae00b7d85b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:18.056553 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3244f6-5745-4daf-8348-5dae00b7d85b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:18.056858 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3244f6-5745-4daf-8348-5dae00b7d85b None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:18.057495 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3244f6-5745-4daf-8348-5dae00b7d85b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:18.057790 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3244f6-5745-4daf-8348-5dae00b7d85b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:18.059156 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3244f6-5745-4daf-8348-5dae00b7d85b None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:18.070925 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 268/519] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:03:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:18.071671 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3244f6-5745-4daf-8348-5dae00b7d85b None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:18.105389 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 252/520] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 6 16:03:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 16:03:18.106219 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c61957e-6789-48fb-8b6d-e4b328d5f1ed None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:18.160886 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:18.167151 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3244f6-5745-4daf-8348-5dae00b7d85b None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:18.199934 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1004fc96-4dfe-47f0-a2f3-3d53622a9f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:18.200543 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1004fc96-4dfe-47f0-a2f3-3d53622a9f41 None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:18.200871 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1004fc96-4dfe-47f0-a2f3-3d53622a9f41 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:18.201139 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1004fc96-4dfe-47f0-a2f3-3d53622a9f41 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:18.201774 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1004fc96-4dfe-47f0-a2f3-3d53622a9f41 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:18.203212 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c61957e-6789-48fb-8b6d-e4b328d5f1ed None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:18.203328 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1004fc96-4dfe-47f0-a2f3-3d53622a9f41 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:18.225901 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 269/521] 10.222.0.43 () {60 vars in 1132 bytes} [Tue Sep 6 16:03:18 2022] POST /identity/v3/users => generated 247 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:03:18.226723 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1004fc96-4dfe-47f0-a2f3-3d53622a9f41 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:18.239395 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e43a0ec-b1fb-4bfe-8252-69c13bb75959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:18.240053 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e43a0ec-b1fb-4bfe-8252-69c13bb75959 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:18.240367 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e43a0ec-b1fb-4bfe-8252-69c13bb75959 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:18.240636 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e43a0ec-b1fb-4bfe-8252-69c13bb75959 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:18.253461 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e43a0ec-b1fb-4bfe-8252-69c13bb75959 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:18.255431 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e43a0ec-b1fb-4bfe-8252-69c13bb75959 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:18.268300 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-5e43a0ec-b1fb-4bfe-8252-69c13bb75959 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:03:18.268300 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:18.268300 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:18.268300 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:18.268300 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:18.268300 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:18.268300 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:18.268300 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:18.268300 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:18.268300 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:18.268300 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:18.268300 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:18.268300 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:03:18.268300 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:03:18.268300 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:03:18.269327 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:03:18.269327 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:18.269327 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:18.269327 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:18.269327 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:18.269327 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:18.269327 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:18.269327 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:18.269327 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:18.269327 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:18.269327 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:18.269327 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:18.269327 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:18.269327 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:18.269327 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:18.269327 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:18.269327 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:18.270237 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:03:18.270237 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:03:18.270237 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:03:18.270237 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:03:18.270237 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:03:18.270237 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:03:18.270237 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:03:18.270237 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:18.271720 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:18.271796 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 253/522] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 6 16:03:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:18.271796 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e43a0ec-b1fb-4bfe-8252-69c13bb75959 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:18.347473 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1004fc96-4dfe-47f0-a2f3-3d53622a9f41 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:18.382146 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e2414c-cb93-4075-b729-1ae4f298f797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:18.382845 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e2414c-cb93-4075-b729-1ae4f298f797 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:18.383212 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e2414c-cb93-4075-b729-1ae4f298f797 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:18.383558 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e2414c-cb93-4075-b729-1ae4f298f797 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:18.384271 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e2414c-cb93-4075-b729-1ae4f298f797 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:18.384571 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e2414c-cb93-4075-b729-1ae4f298f797 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:18.386048 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e2414c-cb93-4075-b729-1ae4f298f797 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:18.395200 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e43a0ec-b1fb-4bfe-8252-69c13bb75959 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:18.397951 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 270/523] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:03:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:18.398868 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e2414c-cb93-4075-b729-1ae4f298f797 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:18.416261 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:18.438336 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736959fe-93e1-44b9-aeea-7868da9b26c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:18.438976 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736959fe-93e1-44b9-aeea-7868da9b26c2 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:18.439253 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736959fe-93e1-44b9-aeea-7868da9b26c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:18.439576 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736959fe-93e1-44b9-aeea-7868da9b26c2 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:18.447827 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736959fe-93e1-44b9-aeea-7868da9b26c2 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:18.449366 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736959fe-93e1-44b9-aeea-7868da9b26c2 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:18.461419 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-736959fe-93e1-44b9-aeea-7868da9b26c2 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 06 16:03:18.461419 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:18.461419 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:18.461419 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:18.461419 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:18.461419 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:18.461419 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:18.461419 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:18.461419 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:18.461419 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:18.461419 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:18.461419 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:18.461419 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:03:18.461419 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:03:18.461419 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:03:18.461419 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:03:18.462449 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:18.462449 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:18.462449 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:03:18.462449 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:03:18.462449 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:03:18.462449 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:03:18.462449 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:18.462449 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:18.462449 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:18.462449 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:18.462449 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:18.462449 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:18.462449 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:18.462449 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:18.462449 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:18.462449 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:18.462449 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:18.462449 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:18.463330 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:18.463330 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:18.463330 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:03:18.463330 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:03:18.463330 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:03:18.463330 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:03:18.463330 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:03:18.463330 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:03:18.463330 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 06 16:03:18.463330 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:18.463330 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 254/524] 10.222.0.43 () {56 vars in 1102 bytes} [Tue Sep 6 16:03:18 2022] GET /identity/v3/users/neutron => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:18.463330 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736959fe-93e1-44b9-aeea-7868da9b26c2 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:18.507666 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:18.514998 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e2414c-cb93-4075-b729-1ae4f298f797 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:18.549600 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843e7808-58fe-4557-9201-29b0ef0fb04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:18.550325 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843e7808-58fe-4557-9201-29b0ef0fb04e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:18.550604 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843e7808-58fe-4557-9201-29b0ef0fb04e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:18.550864 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843e7808-58fe-4557-9201-29b0ef0fb04e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:18.562116 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843e7808-58fe-4557-9201-29b0ef0fb04e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:18.563567 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843e7808-58fe-4557-9201-29b0ef0fb04e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:18.567109 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736959fe-93e1-44b9-aeea-7868da9b26c2 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:18.578359 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-843e7808-58fe-4557-9201-29b0ef0fb04e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:18.578359 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:18.578359 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:18.578359 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:18.578359 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:18.578359 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:18.578359 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:18.578359 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:18.578359 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:18.578359 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:18.578359 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:18.578359 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:18.578359 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:03:18.578359 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:03:18.578359 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:03:18.578359 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:03:18.579497 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:18.579497 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:18.579497 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:18.579497 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:18.579497 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:18.579497 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:18.579497 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:18.579497 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:18.579497 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:18.579497 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:18.579497 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:18.579497 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:18.579497 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:18.579497 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:18.579497 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:18.579497 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:18.579497 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:03:18.580838 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:03:18.580838 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:18.580838 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:18.580838 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 271/525] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:03:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:18.580838 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843e7808-58fe-4557-9201-29b0ef0fb04e None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:18.612173 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a92f6f-b65b-494f-9794-019cca92263a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:18.612740 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a92f6f-b65b-494f-9794-019cca92263a None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:18.612980 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a92f6f-b65b-494f-9794-019cca92263a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:18.613274 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a92f6f-b65b-494f-9794-019cca92263a None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:18.614071 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a92f6f-b65b-494f-9794-019cca92263a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:18.614509 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a92f6f-b65b-494f-9794-019cca92263a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:18.615817 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a92f6f-b65b-494f-9794-019cca92263a None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:18.633844 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 255/526] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 6 16:03:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:18.634660 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a92f6f-b65b-494f-9794-019cca92263a None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:18.669253 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:18.693944 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843e7808-58fe-4557-9201-29b0ef0fb04e None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:18.731786 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a18f6-ee35-48e8-8062-af347cb31ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:18.732454 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a18f6-ee35-48e8-8062-af347cb31ccd None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:18.732742 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a18f6-ee35-48e8-8062-af347cb31ccd None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:18.733050 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a18f6-ee35-48e8-8062-af347cb31ccd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:18.740530 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20a18f6-ee35-48e8-8062-af347cb31ccd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:18.741790 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a92f6f-b65b-494f-9794-019cca92263a None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:18.742235 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20a18f6-ee35-48e8-8062-af347cb31ccd None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:18.756150 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-e20a18f6-ee35-48e8-8062-af347cb31ccd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:03:18.756150 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:18.756150 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:18.756150 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:18.756150 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:18.756150 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:18.756150 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:18.756150 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:18.756150 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:18.756150 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:18.756150 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:18.756150 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:18.756150 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:03:18.756150 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:03:18.756150 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:03:18.757073 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:03:18.757073 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:18.757073 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:18.757073 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:18.757073 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:18.757073 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:18.757073 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:18.757073 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:18.757073 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:18.757073 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:18.757073 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:18.757073 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:18.757073 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:18.757073 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:18.757073 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:18.757073 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:18.757073 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:18.758784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:03:18.758784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:03:18.758784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:03:18.758784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:03:18.758784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:03:18.758784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:03:18.758784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:03:18.758784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:18.758784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 272/527] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 6 16:03:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:18.758784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a18f6-ee35-48e8-8062-af347cb31ccd None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:18.770844 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:18.784138 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1de8da-1da9-482a-8921-24bdc137c8c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:18.784781 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1de8da-1da9-482a-8921-24bdc137c8c2 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:18.785101 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1de8da-1da9-482a-8921-24bdc137c8c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:18.785321 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1de8da-1da9-482a-8921-24bdc137c8c2 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:18.786114 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1de8da-1da9-482a-8921-24bdc137c8c2 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:18.786417 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1de8da-1da9-482a-8921-24bdc137c8c2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:18.787878 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1de8da-1da9-482a-8921-24bdc137c8c2 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:18.828791 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 256/528] 10.222.0.43 () {56 vars in 1139 bytes} [Tue Sep 6 16:03:18 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:18.829635 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1de8da-1da9-482a-8921-24bdc137c8c2 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:18.867182 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a18f6-ee35-48e8-8062-af347cb31ccd None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:18.899704 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d978eccd-d91f-4b44-9806-02a795c3ca81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:18.900312 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d978eccd-d91f-4b44-9806-02a795c3ca81 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:18.900605 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d978eccd-d91f-4b44-9806-02a795c3ca81 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:18.900920 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d978eccd-d91f-4b44-9806-02a795c3ca81 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:18.901566 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d978eccd-d91f-4b44-9806-02a795c3ca81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:18.901869 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d978eccd-d91f-4b44-9806-02a795c3ca81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:18.903262 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d978eccd-d91f-4b44-9806-02a795c3ca81 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:18.915193 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 273/529] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:03:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:18.915982 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d978eccd-d91f-4b44-9806-02a795c3ca81 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:18.931571 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:18.941909 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1de8da-1da9-482a-8921-24bdc137c8c2 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:18.978659 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d39fc6f-a243-4267-a907-2aecc322a2c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:18.979488 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d39fc6f-a243-4267-a907-2aecc322a2c6 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:18.980009 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d39fc6f-a243-4267-a907-2aecc322a2c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:18.980418 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d39fc6f-a243-4267-a907-2aecc322a2c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:18.981000 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:18.982509 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d39fc6f-a243-4267-a907-2aecc322a2c6 None admin] RBAC: Adding query filter params (role.id=31321627b02c4ff188f8fe1521ce3181, scope.project.id=128d210e8dad48ac921ed4cb1d444ceb, user.id=2c163b39e61648c385f065ce8091c3ab) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:18.982975 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d39fc6f-a243-4267-a907-2aecc322a2c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:18.984629 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d39fc6f-a243-4267-a907-2aecc322a2c6 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:18.991115 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 257/530] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 6 16:03:18 2022] GET /identity/v3/role_assignments?user.id=2c163b39e61648c385f065ce8091c3ab&scope.project.id=128d210e8dad48ac921ed4cb1d444ceb&role.id=31321627b02c4ff188f8fe1521ce3181 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:18.992162 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d39fc6f-a243-4267-a907-2aecc322a2c6 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:19.030205 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d978eccd-d91f-4b44-9806-02a795c3ca81 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:19.070154 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6334afd-5c20-41f3-9ef2-f79daeebf136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:19.070870 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6334afd-5c20-41f3-9ef2-f79daeebf136 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:19.071215 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6334afd-5c20-41f3-9ef2-f79daeebf136 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:19.071548 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6334afd-5c20-41f3-9ef2-f79daeebf136 None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:19.072348 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6334afd-5c20-41f3-9ef2-f79daeebf136 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:19.072711 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6334afd-5c20-41f3-9ef2-f79daeebf136 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:19.074333 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6334afd-5c20-41f3-9ef2-f79daeebf136 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:19.099462 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 274/531] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 6 16:03:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:19.099462 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6334afd-5c20-41f3-9ef2-f79daeebf136 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:19.139867 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d39fc6f-a243-4267-a907-2aecc322a2c6 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:19.184349 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81711747-55fc-40ae-b120-6f162d5d4e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:19.185313 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81711747-55fc-40ae-b120-6f162d5d4e72 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:19.185733 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81711747-55fc-40ae-b120-6f162d5d4e72 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:19.186141 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81711747-55fc-40ae-b120-6f162d5d4e72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:19.216434 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6334afd-5c20-41f3-9ef2-f79daeebf136 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:19.219227 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:19.248344 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:19.261442 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f41b4c-ae20-4802-a63a-d73ed8e8f851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:19.262226 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f41b4c-ae20-4802-a63a-d73ed8e8f851 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:19.262505 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f41b4c-ae20-4802-a63a-d73ed8e8f851 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:19.262776 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f41b4c-ae20-4802-a63a-d73ed8e8f851 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:19.271757 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f41b4c-ae20-4802-a63a-d73ed8e8f851 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:19.274117 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f41b4c-ae20-4802-a63a-d73ed8e8f851 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:19.279285 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81711747-55fc-40ae-b120-6f162d5d4e72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:19.282339 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81711747-55fc-40ae-b120-6f162d5d4e72 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:19.293882 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-91f41b4c-ae20-4802-a63a-d73ed8e8f851 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:03:19.293882 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:19.293882 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:19.293882 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:19.293882 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:19.293882 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:19.293882 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:19.293882 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:19.293882 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:19.293882 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:19.293882 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:19.293882 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:19.293882 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:03:19.293882 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:03:19.293882 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:03:19.294764 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:03:19.294764 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:19.294764 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:19.294764 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:19.294764 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:19.294764 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:19.294764 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:19.294764 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:19.294764 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:19.294764 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:19.294764 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:19.294764 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:19.294764 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:19.294764 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:19.294764 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:19.294764 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:19.294764 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:19.295598 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:03:19.295598 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:03:19.295598 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:03:19.295598 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:03:19.295598 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:03:19.295598 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:03:19.295598 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:03:19.295598 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:19.295598 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 275/532] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 6 16:03:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:19.296783 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f41b4c-ae20-4802-a63a-d73ed8e8f851 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:19.422952 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 258/533] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 6 16:03:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 16:03:19.425194 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81711747-55fc-40ae-b120-6f162d5d4e72 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:19.433761 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f41b4c-ae20-4802-a63a-d73ed8e8f851 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:19.479998 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fff4e21-d55f-4a5b-9de2-c2ae4e56923c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:19.480703 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fff4e21-d55f-4a5b-9de2-c2ae4e56923c None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:19.481095 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fff4e21-d55f-4a5b-9de2-c2ae4e56923c None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:19.481369 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fff4e21-d55f-4a5b-9de2-c2ae4e56923c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:19.482558 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fff4e21-d55f-4a5b-9de2-c2ae4e56923c None admin] RBAC: Adding query filter params (role.id=31321627b02c4ff188f8fe1521ce3181, scope.project.id=128d210e8dad48ac921ed4cb1d444ceb, user.id=63de41f4348243998029f7d07aa5dc40) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:19.482870 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fff4e21-d55f-4a5b-9de2-c2ae4e56923c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:19.484599 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fff4e21-d55f-4a5b-9de2-c2ae4e56923c None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:19.491931 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 276/534] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 6 16:03:19 2022] GET /identity/v3/role_assignments?user.id=63de41f4348243998029f7d07aa5dc40&scope.project.id=128d210e8dad48ac921ed4cb1d444ceb&role.id=31321627b02c4ff188f8fe1521ce3181 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:19.492819 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fff4e21-d55f-4a5b-9de2-c2ae4e56923c None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:19.501616 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:19.560658 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:19.623349 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fff4e21-d55f-4a5b-9de2-c2ae4e56923c None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:19.627737 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81711747-55fc-40ae-b120-6f162d5d4e72 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:19.663021 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978ab5f5-aa8a-4470-abf3-deae4aaad5ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:19.663674 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ab5f5-aa8a-4470-abf3-deae4aaad5ab None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:19.663948 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ab5f5-aa8a-4470-abf3-deae4aaad5ab None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:19.664217 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ab5f5-aa8a-4470-abf3-deae4aaad5ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:19.679195 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978ab5f5-aa8a-4470-abf3-deae4aaad5ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:19.680603 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978ab5f5-aa8a-4470-abf3-deae4aaad5ab None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:19.697821 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-978ab5f5-aa8a-4470-abf3-deae4aaad5ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:19.697821 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:19.697821 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:19.697821 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:19.697821 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:19.697821 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:19.697821 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:19.697821 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:19.697821 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:19.697821 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:19.697821 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:19.697821 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:19.697821 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:03:19.697821 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:03:19.697821 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:03:19.697821 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:03:19.699228 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:19.699228 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:19.699228 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:19.699228 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:19.699228 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:19.699228 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:19.699228 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:19.699228 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:19.699228 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:19.699228 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:19.699228 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:19.699228 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:19.699228 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:19.699228 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:19.699228 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:19.699228 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:19.699228 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:03:19.700054 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:03:19.700054 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:19.700054 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:19.704330 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 277/535] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:03:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:19.706261 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978ab5f5-aa8a-4470-abf3-deae4aaad5ab None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:19.716021 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f55ff4d-ee7a-4001-9406-1fcc46ff2896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:19.716647 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f55ff4d-ee7a-4001-9406-1fcc46ff2896 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:19.716973 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f55ff4d-ee7a-4001-9406-1fcc46ff2896 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:19.717252 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f55ff4d-ee7a-4001-9406-1fcc46ff2896 None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:19.717964 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f55ff4d-ee7a-4001-9406-1fcc46ff2896 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:19.718274 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f55ff4d-ee7a-4001-9406-1fcc46ff2896 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:19.722014 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f55ff4d-ee7a-4001-9406-1fcc46ff2896 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:19.741144 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 259/536] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 6 16:03:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:19.743053 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f55ff4d-ee7a-4001-9406-1fcc46ff2896 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:19.862998 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978ab5f5-aa8a-4470-abf3-deae4aaad5ab None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:19.864729 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f55ff4d-ee7a-4001-9406-1fcc46ff2896 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:19.866319 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:19.883862 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:19.908404 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e026cea-204e-467d-b7c5-8c1d5c943583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:19.909095 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e026cea-204e-467d-b7c5-8c1d5c943583 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:19.909376 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e026cea-204e-467d-b7c5-8c1d5c943583 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:19.909637 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e026cea-204e-467d-b7c5-8c1d5c943583 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:19.910552 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e026cea-204e-467d-b7c5-8c1d5c943583 None admin] RBAC: Adding query filter params (role.id=31321627b02c4ff188f8fe1521ce3181, scope.project.id=128d210e8dad48ac921ed4cb1d444ceb, user.id=d1b05748e4284153b0728164cd508dc9) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:19.910765 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e026cea-204e-467d-b7c5-8c1d5c943583 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:19.912770 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e026cea-204e-467d-b7c5-8c1d5c943583 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:19.919660 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 260/537] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 6 16:03:19 2022] GET /identity/v3/role_assignments?user.id=d1b05748e4284153b0728164cd508dc9&scope.project.id=128d210e8dad48ac921ed4cb1d444ceb&role.id=31321627b02c4ff188f8fe1521ce3181 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:03:19.944688 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb974a81-8802-4601-8cf0-37fdda1f7f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:19.945749 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb974a81-8802-4601-8cf0-37fdda1f7f78 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:19.946233 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb974a81-8802-4601-8cf0-37fdda1f7f78 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:19.946692 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb974a81-8802-4601-8cf0-37fdda1f7f78 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:19.947583 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb974a81-8802-4601-8cf0-37fdda1f7f78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:19.948090 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb974a81-8802-4601-8cf0-37fdda1f7f78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:19.949769 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb974a81-8802-4601-8cf0-37fdda1f7f78 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:19.974945 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 278/538] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:03:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:03:19.980530 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e026cea-204e-467d-b7c5-8c1d5c943583 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:20.122062 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e026cea-204e-467d-b7c5-8c1d5c943583 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:20.157331 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bbcbb5-22e2-418f-a8b5-1691fe4904fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:20.158004 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bbcbb5-22e2-418f-a8b5-1691fe4904fb None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:20.158283 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bbcbb5-22e2-418f-a8b5-1691fe4904fb None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:20.158558 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bbcbb5-22e2-418f-a8b5-1691fe4904fb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:20.167065 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bbcbb5-22e2-418f-a8b5-1691fe4904fb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:20.168649 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bbcbb5-22e2-418f-a8b5-1691fe4904fb None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:20.182019 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-d5bbcbb5-22e2-418f-a8b5-1691fe4904fb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:03:20.182019 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:20.182019 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:20.182019 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:20.182019 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:20.182019 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:20.182019 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:20.182019 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:20.182019 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:20.182019 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:20.182019 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:20.182019 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:20.182019 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:03:20.182019 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:03:20.182019 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:03:20.182923 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:03:20.182923 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:20.182923 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:20.182923 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:20.182923 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:20.182923 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:20.182923 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:20.182923 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:20.182923 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:20.182923 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:20.182923 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:20.182923 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:20.182923 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:20.182923 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:20.182923 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:20.182923 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:20.182923 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:20.183755 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:03:20.183755 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:03:20.183755 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:03:20.183755 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:03:20.183755 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:03:20.183755 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:03:20.183755 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:03:20.183755 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:20.183755 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 261/539] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 6 16:03:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:20.189351 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb974a81-8802-4601-8cf0-37fdda1f7f78 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:20.240357 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adefc68c-86ca-496c-85d7-0b71ee83eeb3 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:20.240656 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adefc68c-86ca-496c-85d7-0b71ee83eeb3 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:20.240854 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adefc68c-86ca-496c-85d7-0b71ee83eeb3 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:20.242894 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 262/540] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:03:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:03:20.250331 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a19fcd9-5ab5-4137-afdc-e25b36f48998 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:20.250331 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a19fcd9-5ab5-4137-afdc-e25b36f48998 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:20.250331 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a19fcd9-5ab5-4137-afdc-e25b36f48998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:20.286366 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-0a19fcd9-5ab5-4137-afdc-e25b36f48998 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:03:20.294786 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0a19fcd9-5ab5-4137-afdc-e25b36f48998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:03:20.349128 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:20.378307 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb974a81-8802-4601-8cf0-37fdda1f7f78 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:20.416908 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 263/541] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:03:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:03:20.433319 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4832870-9e52-40a6-9e20-795f36f42ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:20.433931 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4832870-9e52-40a6-9e20-795f36f42ddc None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:20.434201 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4832870-9e52-40a6-9e20-795f36f42ddc None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:20.434474 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4832870-9e52-40a6-9e20-795f36f42ddc None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:20.435170 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4832870-9e52-40a6-9e20-795f36f42ddc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:20.435474 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4832870-9e52-40a6-9e20-795f36f42ddc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:20.436948 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4832870-9e52-40a6-9e20-795f36f42ddc None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:20.448950 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 279/542] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 6 16:03:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:20.457959 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a19fcd9-5ab5-4137-afdc-e25b36f48998 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:20.565578 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a19fcd9-5ab5-4137-afdc-e25b36f48998 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:20.600673 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df63759-93da-4ff5-b5df-f465326246ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:20.601313 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df63759-93da-4ff5-b5df-f465326246ea None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:20.601591 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df63759-93da-4ff5-b5df-f465326246ea None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:20.601860 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df63759-93da-4ff5-b5df-f465326246ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:20.602606 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df63759-93da-4ff5-b5df-f465326246ea None admin] RBAC: Adding query filter params (role.id=31321627b02c4ff188f8fe1521ce3181, scope.project.id=128d210e8dad48ac921ed4cb1d444ceb, user.id=10565aefeaaa40549ed38bb9e0f4b189) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:20.602913 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df63759-93da-4ff5-b5df-f465326246ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:20.604385 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df63759-93da-4ff5-b5df-f465326246ea None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:20.610618 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 264/543] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 6 16:03:20 2022] GET /identity/v3/role_assignments?user.id=10565aefeaaa40549ed38bb9e0f4b189&scope.project.id=128d210e8dad48ac921ed4cb1d444ceb&role.id=31321627b02c4ff188f8fe1521ce3181 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:20.796690 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4832870-9e52-40a6-9e20-795f36f42ddc None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:20.831567 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:20.907506 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4832870-9e52-40a6-9e20-795f36f42ddc None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:20.951784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a1a7aa-2982-43b9-8889-a95eaed62f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:20.952422 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a1a7aa-2982-43b9-8889-a95eaed62f3f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:20.952699 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a1a7aa-2982-43b9-8889-a95eaed62f3f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:20.953038 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a1a7aa-2982-43b9-8889-a95eaed62f3f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:20.969632 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a1a7aa-2982-43b9-8889-a95eaed62f3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:03:20.974955 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-57a1a7aa-2982-43b9-8889-a95eaed62f3f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:03:20.974955 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:20.974955 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:20.974955 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:20.974955 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:20.974955 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:20.974955 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:20.974955 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:20.974955 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:20.974955 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:20.974955 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:20.974955 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:20.974955 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:03:20.974955 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:03:20.974955 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:03:20.974955 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:03:20.975822 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:03:20.975822 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:03:20.975822 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:03:20.975822 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:03:20.975822 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:20.975822 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:20.975822 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:20.975822 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:20.975822 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:20.975822 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:20.975822 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:20.975822 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:20.975822 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:20.975822 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:20.975822 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:20.975822 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:20.976615 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:20.976615 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:20.976615 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:20.976615 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:20.976615 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:03:20.976615 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:03:20.976615 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:03:20.976615 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:03:20.976615 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:03:20.976615 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:03:20.976615 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:03:20.976615 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:20.978698 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 280/544] 10.222.0.43 () {56 vars in 1102 bytes} [Tue Sep 6 16:03:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:20.984522 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df63759-93da-4ff5-b5df-f465326246ea None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:21.058082 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:21.112997 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9e702b-d9ea-4fc3-8c0b-2e45e069661d None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:21.113184 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9e702b-d9ea-4fc3-8c0b-2e45e069661d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:21.113438 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9e702b-d9ea-4fc3-8c0b-2e45e069661d None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:21.113650 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df63759-93da-4ff5-b5df-f465326246ea None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:21.119013 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 281/545] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:03:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:03:21.122206 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1361eb9f-f3fa-4495-9777-2b5e0b957a21 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:21.122489 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1361eb9f-f3fa-4495-9777-2b5e0b957a21 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:21.122765 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1361eb9f-f3fa-4495-9777-2b5e0b957a21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:21.152080 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c33f7b-1449-4598-8e44-2a77cfcc3f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:21.152783 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c33f7b-1449-4598-8e44-2a77cfcc3f78 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:21.153072 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c33f7b-1449-4598-8e44-2a77cfcc3f78 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:21.153402 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c33f7b-1449-4598-8e44-2a77cfcc3f78 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:21.154094 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c33f7b-1449-4598-8e44-2a77cfcc3f78 None admin] RBAC: Adding query filter params (name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:21.154391 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c33f7b-1449-4598-8e44-2a77cfcc3f78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:21.155804 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c33f7b-1449-4598-8e44-2a77cfcc3f78 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:21.165751 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-1361eb9f-f3fa-4495-9777-2b5e0b957a21 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:03:21.167018 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-1361eb9f-f3fa-4495-9777-2b5e0b957a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:03:21.174385 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 265/546] 10.222.0.43 () {56 vars in 1103 bytes} [Tue Sep 6 16:03:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:21.179111 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c33f7b-1449-4598-8e44-2a77cfcc3f78 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:21.291554 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 282/547] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:03:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:03:21.337121 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:21.344358 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c33f7b-1449-4598-8e44-2a77cfcc3f78 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:21.392282 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ddea54-3be4-4489-af53-a8ee001da288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:21.392874 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ddea54-3be4-4489-af53-a8ee001da288 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:21.393252 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ddea54-3be4-4489-af53-a8ee001da288 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:21.393522 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ddea54-3be4-4489-af53-a8ee001da288 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:21.404349 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ddea54-3be4-4489-af53-a8ee001da288 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:21.405855 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ddea54-3be4-4489-af53-a8ee001da288 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:21.420088 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-26ddea54-3be4-4489-af53-a8ee001da288 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:21.420088 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:21.420088 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:21.420088 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:21.420088 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:21.420088 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:21.420088 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:21.420088 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:21.420088 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:21.420088 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:21.420088 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:21.420088 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:21.420088 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:03:21.420088 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:03:21.420088 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:03:21.420088 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:03:21.421179 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:21.421179 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:21.421179 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:21.421179 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:21.421179 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:21.421179 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:21.421179 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:21.421179 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:21.421179 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:21.421179 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:21.421179 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:21.421179 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:21.421179 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:21.421179 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:21.421179 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:21.421179 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:21.421179 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:03:21.422208 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:03:21.422208 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:21.422208 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:21.422208 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 266/548] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:03:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:21.427332 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1361eb9f-f3fa-4495-9777-2b5e0b957a21 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:21.459077 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:21.565086 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ddea54-3be4-4489-af53-a8ee001da288 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:21.593095 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1361eb9f-f3fa-4495-9777-2b5e0b957a21 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:21.664529 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8834b8b9-2d3e-42eb-a90d-57ea73452619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:21.665087 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8834b8b9-2d3e-42eb-a90d-57ea73452619 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:21.665365 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8834b8b9-2d3e-42eb-a90d-57ea73452619 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:21.665639 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8834b8b9-2d3e-42eb-a90d-57ea73452619 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:21.666298 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8834b8b9-2d3e-42eb-a90d-57ea73452619 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:21.666598 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8834b8b9-2d3e-42eb-a90d-57ea73452619 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:21.667988 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8834b8b9-2d3e-42eb-a90d-57ea73452619 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:21.696575 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 283/549] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:03:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:21.698927 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8834b8b9-2d3e-42eb-a90d-57ea73452619 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:21.746882 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ddea54-3be4-4489-af53-a8ee001da288 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:21.812944 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7eba8f-4cf7-48e1-a8a1-840ce244b998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:21.813591 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7eba8f-4cf7-48e1-a8a1-840ce244b998 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:21.814541 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7eba8f-4cf7-48e1-a8a1-840ce244b998 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:21.815065 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7eba8f-4cf7-48e1-a8a1-840ce244b998 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:21.838685 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7eba8f-4cf7-48e1-a8a1-840ce244b998 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:03:21.843315 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:21.844015 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-cf7eba8f-4cf7-48e1-a8a1-840ce244b998 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:03:21.844015 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:21.844015 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:21.844015 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:21.844015 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:21.844015 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:21.844015 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:21.844015 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:21.844015 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:21.844015 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:21.844015 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:21.844015 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:21.844015 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:03:21.844015 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:03:21.844015 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:03:21.844015 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:03:21.845499 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:03:21.845499 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:03:21.845499 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:03:21.845499 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:03:21.845499 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:21.845499 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:21.845499 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:21.845499 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:21.845499 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:21.845499 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:21.845499 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:21.845499 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:21.845499 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:21.845499 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:21.845499 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:21.845499 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:21.846458 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:21.846458 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:21.846458 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:21.846458 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:21.846458 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:03:21.846458 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:03:21.846458 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:03:21.846458 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:03:21.846458 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:03:21.846458 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:03:21.846458 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:03:21.846458 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:21.846458 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 267/550] 10.222.0.43 () {56 vars in 1102 bytes} [Tue Sep 6 16:03:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:03:21.851288 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7eba8f-4cf7-48e1-a8a1-840ce244b998 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:21.867841 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:21.898637 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8834b8b9-2d3e-42eb-a90d-57ea73452619 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:21.966759 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051ad2ad-9c4c-4089-bf37-157621b4807f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:21.967322 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051ad2ad-9c4c-4089-bf37-157621b4807f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:21.967597 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051ad2ad-9c4c-4089-bf37-157621b4807f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:21.967865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051ad2ad-9c4c-4089-bf37-157621b4807f None admin] PATH_INFO: `/v3/users/swift` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:21.978812 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051ad2ad-9c4c-4089-bf37-157621b4807f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:21.980257 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051ad2ad-9c4c-4089-bf37-157621b4807f None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:21.994132 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-051ad2ad-9c4c-4089-bf37-157621b4807f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 06 16:03:21.994132 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:21.994132 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:21.994132 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:21.994132 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:21.994132 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:21.994132 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:21.994132 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:21.994132 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:21.994132 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:21.994132 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:21.994132 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:21.994132 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:03:21.994132 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:03:21.994132 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:03:21.994132 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:03:21.995886 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:21.995886 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:21.995886 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:03:21.995886 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:03:21.995886 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:03:21.995886 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:03:21.995886 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:21.995886 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:21.995886 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:21.995886 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:21.995886 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:21.995886 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:21.995886 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:21.995886 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:21.995886 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:21.995886 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:21.995886 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:21.995886 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:21.996998 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:21.996998 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:21.996998 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:03:21.996998 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:03:21.996998 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:03:21.996998 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:03:21.996998 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:03:21.996998 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:03:21.996998 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 06 16:03:21.996998 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:21.996998 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 284/551] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 6 16:03:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:22.002003 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051ad2ad-9c4c-4089-bf37-157621b4807f None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:22.027072 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7eba8f-4cf7-48e1-a8a1-840ce244b998 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:22.070575 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1a16a9-7d23-4921-b205-13a2dafa1eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:22.071200 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1a16a9-7d23-4921-b205-13a2dafa1eef None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:22.071797 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1a16a9-7d23-4921-b205-13a2dafa1eef None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:22.072074 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1a16a9-7d23-4921-b205-13a2dafa1eef None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:22.073036 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1a16a9-7d23-4921-b205-13a2dafa1eef None admin] RBAC: Adding query filter params (name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:22.073339 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1a16a9-7d23-4921-b205-13a2dafa1eef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:22.075471 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1a16a9-7d23-4921-b205-13a2dafa1eef None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:22.091684 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 268/552] 10.222.0.43 () {56 vars in 1103 bytes} [Tue Sep 6 16:03:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:22.093234 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f7f447-c3a0-45b4-b805-4c5a684d2a27 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:22.093523 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f7f447-c3a0-45b4-b805-4c5a684d2a27 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:22.094155 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f7f447-c3a0-45b4-b805-4c5a684d2a27 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:22.096108 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 269/553] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:03:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:03:22.096857 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f7f447-c3a0-45b4-b805-4c5a684d2a27 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:22.123757 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051ad2ad-9c4c-4089-bf37-157621b4807f None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:22.165861 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8c1435-4422-47f0-bf66-57be52a640dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:22.166619 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8c1435-4422-47f0-bf66-57be52a640dd None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:22.166902 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8c1435-4422-47f0-bf66-57be52a640dd None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:22.167172 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8c1435-4422-47f0-bf66-57be52a640dd None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:22.168467 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8c1435-4422-47f0-bf66-57be52a640dd None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:22.168790 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8c1435-4422-47f0-bf66-57be52a640dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:22.170236 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8c1435-4422-47f0-bf66-57be52a640dd None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:22.172695 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:22.176467 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:22.195820 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 285/554] 10.222.0.43 () {56 vars in 1135 bytes} [Tue Sep 6 16:03:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:22.195820 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084a0dcf-ebab-42ac-bae5-09de35f6822d None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:22.195820 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084a0dcf-ebab-42ac-bae5-09de35f6822d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:22.195820 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084a0dcf-ebab-42ac-bae5-09de35f6822d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:22.216198 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-084a0dcf-ebab-42ac-bae5-09de35f6822d None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:03:22.217590 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-084a0dcf-ebab-42ac-bae5-09de35f6822d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:03:22.218149 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f7f447-c3a0-45b4-b805-4c5a684d2a27 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:22.263944 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9334554-2b0b-48a5-9d7e-3e604fa2b2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:22.264584 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9334554-2b0b-48a5-9d7e-3e604fa2b2f2 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:22.264904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9334554-2b0b-48a5-9d7e-3e604fa2b2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:22.265163 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9334554-2b0b-48a5-9d7e-3e604fa2b2f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:22.282316 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9334554-2b0b-48a5-9d7e-3e604fa2b2f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:22.283817 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9334554-2b0b-48a5-9d7e-3e604fa2b2f2 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:22.307220 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-b9334554-2b0b-48a5-9d7e-3e604fa2b2f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:22.307220 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:22.307220 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:22.307220 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:22.307220 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:22.307220 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:22.307220 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:22.307220 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:22.307220 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:22.307220 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:22.307220 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:22.307220 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:22.307220 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:03:22.307220 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:03:22.307220 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:03:22.307220 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:03:22.308367 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:22.308367 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:22.308367 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:22.308367 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:22.308367 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:22.308367 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:22.308367 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:22.308367 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:22.308367 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:22.308367 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:22.308367 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:22.308367 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:22.308367 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:22.308367 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:22.308367 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:22.308367 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:22.308367 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:03:22.309364 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:03:22.309364 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:22.309364 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:22.309364 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 270/555] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:03:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:22.309364 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9334554-2b0b-48a5-9d7e-3e604fa2b2f2 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:22.309960 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 286/556] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:03:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:03:22.311018 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084a0dcf-ebab-42ac-bae5-09de35f6822d None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:22.441507 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084a0dcf-ebab-42ac-bae5-09de35f6822d None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:22.442242 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9334554-2b0b-48a5-9d7e-3e604fa2b2f2 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:22.466413 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:22.478493 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:22.479497 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa88af61-20a4-4c7f-9ae8-d86597d20cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:22.480290 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa88af61-20a4-4c7f-9ae8-d86597d20cd9 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:22.480639 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa88af61-20a4-4c7f-9ae8-d86597d20cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:22.481115 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa88af61-20a4-4c7f-9ae8-d86597d20cd9 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:22.481863 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa88af61-20a4-4c7f-9ae8-d86597d20cd9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:22.482251 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa88af61-20a4-4c7f-9ae8-d86597d20cd9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:22.483714 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa88af61-20a4-4c7f-9ae8-d86597d20cd9 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:22.494259 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864f97d7-e024-4387-80c7-ecc496f067d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:22.495222 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f97d7-e024-4387-80c7-ecc496f067d5 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:22.495764 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f97d7-e024-4387-80c7-ecc496f067d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:22.496199 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f97d7-e024-4387-80c7-ecc496f067d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:22.501957 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 287/557] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:03:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:22.501957 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320d07d4-604b-479b-be87-0942192db9c4 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:22.501957 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320d07d4-604b-479b-be87-0942192db9c4 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:22.501957 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320d07d4-604b-479b-be87-0942192db9c4 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:22.505225 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 288/558] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:03:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:03:22.506425 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee48e90-ddea-4df7-b6f5-bfbd792bd807 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:22.506785 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee48e90-ddea-4df7-b6f5-bfbd792bd807 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:22.507289 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee48e90-ddea-4df7-b6f5-bfbd792bd807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:22.526895 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-fee48e90-ddea-4df7-b6f5-bfbd792bd807 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:03:22.528317 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-fee48e90-ddea-4df7-b6f5-bfbd792bd807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:03:22.587168 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864f97d7-e024-4387-80c7-ecc496f067d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:22.588656 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864f97d7-e024-4387-80c7-ecc496f067d5 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:22.601338 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 289/559] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:03:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:03:22.604811 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee48e90-ddea-4df7-b6f5-bfbd792bd807 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:22.712547 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee48e90-ddea-4df7-b6f5-bfbd792bd807 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:22.721384 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 271/560] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 6 16:03:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 16:03:22.722888 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864f97d7-e024-4387-80c7-ecc496f067d5 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:22.730366 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:22.754770 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:22.762540 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fa249b-23fe-4473-bc2c-200d1fa0668a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:22.763340 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa249b-23fe-4473-bc2c-200d1fa0668a None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:22.763621 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa249b-23fe-4473-bc2c-200d1fa0668a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:22.764198 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa249b-23fe-4473-bc2c-200d1fa0668a None admin] PATH_INFO: `/v3/users/glance` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:22.773053 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fa249b-23fe-4473-bc2c-200d1fa0668a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:22.775033 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fa249b-23fe-4473-bc2c-200d1fa0668a None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:22.790658 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-f1fa249b-23fe-4473-bc2c-200d1fa0668a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 06 16:03:22.790658 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:22.790658 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:22.790658 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:22.790658 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:22.790658 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:22.790658 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:22.790658 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:22.790658 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:22.790658 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:22.790658 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:22.790658 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:22.790658 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:03:22.790658 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:03:22.790658 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:03:22.790658 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:03:22.791686 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:22.791686 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:22.791686 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:03:22.791686 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:03:22.791686 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:03:22.791686 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:03:22.791686 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:22.791686 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:22.791686 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:22.791686 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:22.791686 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:22.791686 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:22.791686 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:22.791686 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:22.791686 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:22.791686 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:22.791686 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:22.791686 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:22.792937 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:22.792937 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:22.792937 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:03:22.792937 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:03:22.792937 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:03:22.792937 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:03:22.792937 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:03:22.792937 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:03:22.792937 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 06 16:03:22.792937 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:22.792937 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 290/561] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 6 16:03:22 2022] GET /identity/v3/users/glance => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:22.792937 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fa249b-23fe-4473-bc2c-200d1fa0668a None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:22.846957 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864f97d7-e024-4387-80c7-ecc496f067d5 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:22.882638 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faddb2c9-31f4-4c3d-84c3-ac8a119e8b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:22.883614 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faddb2c9-31f4-4c3d-84c3-ac8a119e8b78 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:22.883903 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faddb2c9-31f4-4c3d-84c3-ac8a119e8b78 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:22.884172 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faddb2c9-31f4-4c3d-84c3-ac8a119e8b78 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:22.895409 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faddb2c9-31f4-4c3d-84c3-ac8a119e8b78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:03:22.900232 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-faddb2c9-31f4-4c3d-84c3-ac8a119e8b78 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:03:22.900232 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:22.900232 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:22.900232 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:22.900232 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:22.900232 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:22.900232 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:22.900232 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:22.900232 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:22.900232 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:22.900232 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:22.900232 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:22.900232 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:03:22.900232 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:03:22.900232 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:03:22.900232 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:03:22.901261 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:03:22.901261 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:03:22.901261 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:03:22.901261 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:03:22.901261 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:22.901261 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:22.901261 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:22.901261 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:22.901261 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:22.901261 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:22.901261 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:22.901261 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:22.901261 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:22.901261 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:22.901261 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:22.901261 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:22.902076 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:22.902076 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:22.902076 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:22.902076 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:22.902076 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:03:22.902076 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:03:22.902076 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:03:22.902076 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:03:22.902076 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:03:22.902076 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:03:22.902076 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:03:22.902076 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:22.902076 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 272/562] 10.222.0.43 () {56 vars in 1102 bytes} [Tue Sep 6 16:03:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:22.902747 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faddb2c9-31f4-4c3d-84c3-ac8a119e8b78 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:22.907789 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fa249b-23fe-4473-bc2c-200d1fa0668a None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:22.941435 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb70e610-d20b-430a-9db7-dbc57886c81d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:22.942110 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb70e610-d20b-430a-9db7-dbc57886c81d None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:22.942370 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb70e610-d20b-430a-9db7-dbc57886c81d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:22.942642 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb70e610-d20b-430a-9db7-dbc57886c81d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:22.953798 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb70e610-d20b-430a-9db7-dbc57886c81d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:22.955605 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb70e610-d20b-430a-9db7-dbc57886c81d None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:22.969197 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-fb70e610-d20b-430a-9db7-dbc57886c81d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:22.969197 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:22.969197 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:22.969197 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:22.969197 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:22.969197 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:22.969197 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:22.969197 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:22.969197 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:22.969197 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:22.969197 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:22.969197 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:22.969197 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:03:22.969197 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:03:22.969197 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:03:22.969197 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:03:22.970163 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:22.970163 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:22.970163 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:22.970163 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:22.970163 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:22.970163 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:22.970163 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:22.970163 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:22.970163 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:22.970163 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:22.970163 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:22.970163 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:22.970163 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:22.970163 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:22.970163 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:22.970163 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:22.970163 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:03:22.971031 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:03:22.971031 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:22.971031 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:22.971031 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 291/563] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:03:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:22.971533 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a77d2d-82c7-4061-891f-e693c3fe92a3 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:22.971625 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a77d2d-82c7-4061-891f-e693c3fe92a3 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:22.972006 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a77d2d-82c7-4061-891f-e693c3fe92a3 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:22.973187 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 292/564] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:03:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:03:22.973875 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a77d2d-82c7-4061-891f-e693c3fe92a3 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:22.984586 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:22.999256 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:23.005931 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faddb2c9-31f4-4c3d-84c3-ac8a119e8b78 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:23.045104 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df8273d-08f1-4c94-a4dd-38b00535412a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:23.045761 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df8273d-08f1-4c94-a4dd-38b00535412a None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:23.045987 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df8273d-08f1-4c94-a4dd-38b00535412a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:23.046237 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df8273d-08f1-4c94-a4dd-38b00535412a None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:23.046979 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df8273d-08f1-4c94-a4dd-38b00535412a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:23.047272 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df8273d-08f1-4c94-a4dd-38b00535412a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:23.048741 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df8273d-08f1-4c94-a4dd-38b00535412a None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:23.066731 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 273/565] 10.222.0.43 () {56 vars in 1137 bytes} [Tue Sep 6 16:03:22 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:23.067551 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df8273d-08f1-4c94-a4dd-38b00535412a None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:23.085888 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a77d2d-82c7-4061-891f-e693c3fe92a3 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:23.118980 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b815ab22-1dee-43e2-a325-658401c41751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:23.119606 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b815ab22-1dee-43e2-a325-658401c41751 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:23.119886 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b815ab22-1dee-43e2-a325-658401c41751 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:23.120143 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b815ab22-1dee-43e2-a325-658401c41751 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:23.130147 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b815ab22-1dee-43e2-a325-658401c41751 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:03:23.134985 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-b815ab22-1dee-43e2-a325-658401c41751 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:03:23.134985 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:23.134985 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:23.134985 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:23.134985 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:23.134985 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:23.134985 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:23.134985 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:23.134985 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:23.134985 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:23.134985 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:23.134985 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:23.134985 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:03:23.134985 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:03:23.134985 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:03:23.134985 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:03:23.135947 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:03:23.135947 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:03:23.135947 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:03:23.135947 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:03:23.135947 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:23.135947 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:23.135947 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:23.135947 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:23.135947 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:23.135947 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:23.135947 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:23.135947 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:23.135947 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:23.135947 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:23.135947 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:23.135947 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:23.136809 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:23.136809 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:23.136809 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:23.136809 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:23.136809 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:03:23.136809 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:03:23.136809 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:03:23.136809 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:03:23.136809 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:03:23.136809 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:03:23.136809 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:03:23.136809 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:23.136809 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 293/566] 10.222.0.43 () {56 vars in 1102 bytes} [Tue Sep 6 16:03:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:23.137521 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b815ab22-1dee-43e2-a325-658401c41751 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:23.166680 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df8273d-08f1-4c94-a4dd-38b00535412a None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:23.199440 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a4f56b-b098-4fb8-8980-b61b28d8ea54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:23.200232 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a4f56b-b098-4fb8-8980-b61b28d8ea54 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:23.200509 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a4f56b-b098-4fb8-8980-b61b28d8ea54 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:23.200798 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a4f56b-b098-4fb8-8980-b61b28d8ea54 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:23.201453 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a4f56b-b098-4fb8-8980-b61b28d8ea54 None admin] RBAC: Adding query filter params (name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:23.201753 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a4f56b-b098-4fb8-8980-b61b28d8ea54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:23.203133 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a4f56b-b098-4fb8-8980-b61b28d8ea54 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:23.213141 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 274/567] 10.222.0.43 () {56 vars in 1103 bytes} [Tue Sep 6 16:03:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:23.215099 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b0fc30-56ef-4d6e-ac45-2db49f013a55 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:23.215378 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b0fc30-56ef-4d6e-ac45-2db49f013a55 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:23.215650 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b0fc30-56ef-4d6e-ac45-2db49f013a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:23.222192 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:23.225763 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:23.235778 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-37b0fc30-56ef-4d6e-ac45-2db49f013a55 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:03:23.236947 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-37b0fc30-56ef-4d6e-ac45-2db49f013a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:03:23.242230 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b815ab22-1dee-43e2-a325-658401c41751 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:23.283480 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1112fdf-d1ba-4593-80bc-baac272468a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:23.284121 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1112fdf-d1ba-4593-80bc-baac272468a1 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:23.284391 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1112fdf-d1ba-4593-80bc-baac272468a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:23.284655 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1112fdf-d1ba-4593-80bc-baac272468a1 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:23.285395 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1112fdf-d1ba-4593-80bc-baac272468a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:23.285698 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1112fdf-d1ba-4593-80bc-baac272468a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:23.287080 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1112fdf-d1ba-4593-80bc-baac272468a1 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:23.298717 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 294/568] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:03:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:23.299500 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1112fdf-d1ba-4593-80bc-baac272468a1 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:23.310515 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 275/569] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:03:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:03:23.311354 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b0fc30-56ef-4d6e-ac45-2db49f013a55 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:23.402057 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1112fdf-d1ba-4593-80bc-baac272468a1 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:23.413675 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b0fc30-56ef-4d6e-ac45-2db49f013a55 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:23.440527 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50e2ace-1ffd-4a5d-9d37-61578e2eb1a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:23.441471 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50e2ace-1ffd-4a5d-9d37-61578e2eb1a9 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:23.441942 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50e2ace-1ffd-4a5d-9d37-61578e2eb1a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:23.442407 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50e2ace-1ffd-4a5d-9d37-61578e2eb1a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:23.444283 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:23.449919 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6132fb1-12d4-4d7b-a4fb-88575b06dae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:23.450910 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6132fb1-12d4-4d7b-a4fb-88575b06dae4 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:23.451543 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6132fb1-12d4-4d7b-a4fb-88575b06dae4 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:23.451543 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6132fb1-12d4-4d7b-a4fb-88575b06dae4 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:23.452111 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6132fb1-12d4-4d7b-a4fb-88575b06dae4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:23.453096 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6132fb1-12d4-4d7b-a4fb-88575b06dae4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:23.453295 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50e2ace-1ffd-4a5d-9d37-61578e2eb1a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:23.454495 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6132fb1-12d4-4d7b-a4fb-88575b06dae4 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:23.454766 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50e2ace-1ffd-4a5d-9d37-61578e2eb1a9 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:23.457748 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:23.465096 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 276/570] 10.222.0.43 () {56 vars in 1103 bytes} [Tue Sep 6 16:03:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:23.469704 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6132fb1-12d4-4d7b-a4fb-88575b06dae4 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:23.470885 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-c50e2ace-1ffd-4a5d-9d37-61578e2eb1a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:23.470885 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:23.470885 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:23.470885 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:23.470885 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:23.470885 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:23.470885 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:23.470885 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:23.470885 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:23.470885 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:23.470885 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:23.470885 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:23.470885 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:03:23.470885 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:03:23.470885 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:03:23.470885 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:03:23.471802 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:23.471802 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:23.471802 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:23.471802 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:23.471802 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:23.471802 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:23.471802 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:23.471802 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:23.471802 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:23.471802 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:23.471802 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:23.471802 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:23.471802 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:23.471802 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:23.471802 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:23.471802 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:23.471802 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:03:23.472673 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:03:23.472673 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:23.472673 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:23.472673 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 295/571] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:03:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:23.473319 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50e2ace-1ffd-4a5d-9d37-61578e2eb1a9 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:23.574751 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50e2ace-1ffd-4a5d-9d37-61578e2eb1a9 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:23.580686 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6132fb1-12d4-4d7b-a4fb-88575b06dae4 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:23.608262 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5be0d7-d3cc-4996-8f94-08e37e501796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:23.608933 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5be0d7-d3cc-4996-8f94-08e37e501796 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:23.609215 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5be0d7-d3cc-4996-8f94-08e37e501796 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:23.609488 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5be0d7-d3cc-4996-8f94-08e37e501796 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:23.612641 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538af6c9-853d-4b36-95c3-c4714afb897e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:23.613298 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538af6c9-853d-4b36-95c3-c4714afb897e None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:23.613641 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538af6c9-853d-4b36-95c3-c4714afb897e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:23.613914 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538af6c9-853d-4b36-95c3-c4714afb897e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:23.615944 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5be0d7-d3cc-4996-8f94-08e37e501796 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:23.617432 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5be0d7-d3cc-4996-8f94-08e37e501796 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:23.623965 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538af6c9-853d-4b36-95c3-c4714afb897e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:23.625442 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538af6c9-853d-4b36-95c3-c4714afb897e None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:23.628356 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-3c5be0d7-d3cc-4996-8f94-08e37e501796 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:03:23.628356 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:23.628356 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:23.628356 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:23.628356 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:23.628356 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:23.628356 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:23.628356 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:23.628356 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:23.628356 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:23.628356 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:23.628356 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:23.628356 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:03:23.628356 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:03:23.628356 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:03:23.629347 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:03:23.629347 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:23.629347 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:23.629347 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:23.629347 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:23.629347 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:23.629347 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:23.629347 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:23.629347 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:23.629347 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:23.629347 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:23.629347 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:23.629347 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:23.629347 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:23.629347 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:23.629347 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:23.629347 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:23.630299 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:03:23.630299 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:03:23.630299 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:03:23.630299 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:03:23.630299 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:03:23.630299 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:03:23.630299 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:03:23.630299 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:23.630299 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 296/572] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 6 16:03:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:23.630299 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5be0d7-d3cc-4996-8f94-08e37e501796 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:23.642890 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-538af6c9-853d-4b36-95c3-c4714afb897e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:23.642890 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:23.642890 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:23.642890 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:23.642890 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:23.642890 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:23.642890 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:23.642890 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:23.642890 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:23.642890 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:23.642890 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:23.642890 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:23.642890 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:03:23.642890 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:03:23.642890 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:03:23.642890 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:03:23.644201 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:23.644201 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:23.644201 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:23.644201 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:23.644201 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:23.644201 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:23.644201 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:23.644201 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:23.644201 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:23.644201 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:23.644201 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:23.644201 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:23.644201 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:23.644201 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:23.644201 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:23.644201 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:23.644201 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:03:23.645299 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:03:23.645299 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:23.645299 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:23.645299 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 277/573] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:03:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:23.645299 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538af6c9-853d-4b36-95c3-c4714afb897e None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:23.690579 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:23.692083 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:23.734516 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5be0d7-d3cc-4996-8f94-08e37e501796 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:23.746461 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538af6c9-853d-4b36-95c3-c4714afb897e None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:23.773460 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e416506-967d-400a-b5ff-098e921ea330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:23.774094 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e416506-967d-400a-b5ff-098e921ea330 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:23.774372 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e416506-967d-400a-b5ff-098e921ea330 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:23.774643 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e416506-967d-400a-b5ff-098e921ea330 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:23.784015 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd63414a-824d-4f10-a28f-3d3cd4f1e9af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:23.784785 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd63414a-824d-4f10-a28f-3d3cd4f1e9af None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:23.785080 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd63414a-824d-4f10-a28f-3d3cd4f1e9af None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:23.785191 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e416506-967d-400a-b5ff-098e921ea330 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:23.785428 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd63414a-824d-4f10-a28f-3d3cd4f1e9af None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:23.786091 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd63414a-824d-4f10-a28f-3d3cd4f1e9af None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:23.786392 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd63414a-824d-4f10-a28f-3d3cd4f1e9af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:23.786648 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e416506-967d-400a-b5ff-098e921ea330 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:23.787773 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd63414a-824d-4f10-a28f-3d3cd4f1e9af None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:23.799337 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 278/574] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:03:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:23.800124 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd63414a-824d-4f10-a28f-3d3cd4f1e9af None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:23.800826 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-4e416506-967d-400a-b5ff-098e921ea330 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:23.800826 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:23.800826 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:23.800826 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:23.800826 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:23.800826 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:23.800826 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:23.800826 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:23.800826 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:23.800826 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:23.800826 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:23.800826 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:23.800826 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:03:23.800826 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:03:23.800826 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:03:23.800826 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:03:23.801743 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:23.801743 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:23.801743 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:23.801743 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:23.801743 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:23.801743 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:23.801743 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:23.801743 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:23.801743 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:23.801743 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:23.801743 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:23.801743 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:23.801743 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:23.801743 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:23.801743 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:23.801743 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:23.801743 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:03:23.807200 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:03:23.807200 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:23.807200 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:23.807200 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 297/575] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:03:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:23.807200 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e416506-967d-400a-b5ff-098e921ea330 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:23.900125 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e416506-967d-400a-b5ff-098e921ea330 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:23.902218 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd63414a-824d-4f10-a28f-3d3cd4f1e9af None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:23.908453 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:23.909276 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:23.934110 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7680e1-d29a-4300-85de-ef379defe0e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:23.934710 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7680e1-d29a-4300-85de-ef379defe0e9 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:23.934985 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7680e1-d29a-4300-85de-ef379defe0e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:23.935251 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7680e1-d29a-4300-85de-ef379defe0e9 None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:23.935941 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7680e1-d29a-4300-85de-ef379defe0e9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:23.936238 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7680e1-d29a-4300-85de-ef379defe0e9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:23.937637 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d097cf-b82a-4f49-a7a9-0b50abe17dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:23.937761 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7680e1-d29a-4300-85de-ef379defe0e9 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:23.938266 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d097cf-b82a-4f49-a7a9-0b50abe17dde None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:23.938542 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d097cf-b82a-4f49-a7a9-0b50abe17dde None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:23.938811 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d097cf-b82a-4f49-a7a9-0b50abe17dde None admin] PATH_INFO: `/v3/roles/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:23.950008 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 298/576] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 6 16:03:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:23.950138 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d097cf-b82a-4f49-a7a9-0b50abe17dde None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:03:23.950779 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7680e1-d29a-4300-85de-ef379defe0e9 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:23.955206 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-93d097cf-b82a-4f49-a7a9-0b50abe17dde None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:03:23.955206 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:23.955206 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:23.955206 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:23.955206 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:23.955206 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:23.955206 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:23.955206 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:23.955206 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:23.955206 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:23.955206 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:23.955206 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:23.955206 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:03:23.955206 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:03:23.955206 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:03:23.955206 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:03:23.956148 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:03:23.956148 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:03:23.956148 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:03:23.956148 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:03:23.956148 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:23.956148 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:23.956148 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:23.956148 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:23.956148 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:23.956148 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:23.956148 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:23.956148 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:23.956148 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:23.956148 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:23.956148 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:23.956148 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:23.957069 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:23.957069 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:23.957069 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:23.957069 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:23.957069 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:03:23.957069 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:03:23.957069 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:03:23.957069 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:03:23.957069 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:03:23.957069 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:03:23.957069 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:03:23.957069 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:23.957069 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 279/577] 10.222.0.43 () {56 vars in 1102 bytes} [Tue Sep 6 16:03:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:23.957758 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d097cf-b82a-4f49-a7a9-0b50abe17dde None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:24.049395 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7680e1-d29a-4300-85de-ef379defe0e9 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:24.061919 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d097cf-b82a-4f49-a7a9-0b50abe17dde None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:24.087117 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf118e8c-4fd8-40b9-aa9f-55dd44e86c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:24.087726 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf118e8c-4fd8-40b9-aa9f-55dd44e86c04 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:24.088001 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf118e8c-4fd8-40b9-aa9f-55dd44e86c04 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:24.088272 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf118e8c-4fd8-40b9-aa9f-55dd44e86c04 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:24.089323 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf118e8c-4fd8-40b9-aa9f-55dd44e86c04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:24.089323 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf118e8c-4fd8-40b9-aa9f-55dd44e86c04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:24.090616 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf118e8c-4fd8-40b9-aa9f-55dd44e86c04 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:24.102056 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 299/578] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:03:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:24.102837 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf118e8c-4fd8-40b9-aa9f-55dd44e86c04 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:24.103107 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8181d81-a375-408f-b525-1b64891e6a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:24.103812 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8181d81-a375-408f-b525-1b64891e6a71 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:24.104089 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8181d81-a375-408f-b525-1b64891e6a71 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:24.104359 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8181d81-a375-408f-b525-1b64891e6a71 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:24.111245 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8181d81-a375-408f-b525-1b64891e6a71 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:24.112720 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8181d81-a375-408f-b525-1b64891e6a71 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:24.123990 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-b8181d81-a375-408f-b525-1b64891e6a71 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:03:24.123990 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:24.123990 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:24.123990 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:24.123990 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:24.123990 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:24.123990 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:24.123990 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:24.123990 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:24.123990 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:24.123990 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:24.123990 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:24.123990 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:03:24.123990 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:03:24.123990 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:03:24.124918 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:03:24.124918 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:24.124918 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:24.124918 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:24.124918 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:24.124918 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:24.124918 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:24.124918 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:24.124918 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:24.124918 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:24.124918 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:24.124918 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:24.124918 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:24.124918 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:24.124918 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:24.124918 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:24.124918 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:24.125792 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:03:24.125792 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:03:24.125792 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:03:24.125792 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:03:24.125792 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:03:24.125792 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:03:24.125792 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:03:24.125792 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:24.125792 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 280/579] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 6 16:03:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:24.126267 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8181d81-a375-408f-b525-1b64891e6a71 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:24.136946 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:24.153942 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:24.208808 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf118e8c-4fd8-40b9-aa9f-55dd44e86c04 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:24.223208 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8181d81-a375-408f-b525-1b64891e6a71 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:24.242284 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ae352a-4f00-4e25-8297-1ec9f41cddaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:24.245428 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae352a-4f00-4e25-8297-1ec9f41cddaa None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:24.245428 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae352a-4f00-4e25-8297-1ec9f41cddaa None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:24.245428 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae352a-4f00-4e25-8297-1ec9f41cddaa None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:24.245428 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ae352a-4f00-4e25-8297-1ec9f41cddaa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:24.245428 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ae352a-4f00-4e25-8297-1ec9f41cddaa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:24.245793 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ae352a-4f00-4e25-8297-1ec9f41cddaa None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:24.255962 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c42d00-db1c-4ddd-bc46-950366b58d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:24.256568 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c42d00-db1c-4ddd-bc46-950366b58d95 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:24.256869 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c42d00-db1c-4ddd-bc46-950366b58d95 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:24.257136 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c42d00-db1c-4ddd-bc46-950366b58d95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:24.257858 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c42d00-db1c-4ddd-bc46-950366b58d95 None admin] RBAC: Adding query filter params (role.id=31321627b02c4ff188f8fe1521ce3181, scope.project.id=128d210e8dad48ac921ed4cb1d444ceb, user.id=312026c080b64aff9dd483d3591288a6) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:24.258255 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 300/580] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:03:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:24.258379 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c42d00-db1c-4ddd-bc46-950366b58d95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:24.258982 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ae352a-4f00-4e25-8297-1ec9f41cddaa None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:24.259573 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c42d00-db1c-4ddd-bc46-950366b58d95 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:24.266534 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 281/581] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 6 16:03:24 2022] GET /identity/v3/role_assignments?user.id=312026c080b64aff9dd483d3591288a6&scope.project.id=128d210e8dad48ac921ed4cb1d444ceb&role.id=31321627b02c4ff188f8fe1521ce3181 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:24.267322 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c42d00-db1c-4ddd-bc46-950366b58d95 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:24.365285 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:24.379104 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ae352a-4f00-4e25-8297-1ec9f41cddaa None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:24.379476 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:24.391287 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c42d00-db1c-4ddd-bc46-950366b58d95 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:24.420563 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ae785b-6876-42a2-981c-79dcfbe213d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:24.420563 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae785b-6876-42a2-981c-79dcfbe213d4 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:24.420563 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae785b-6876-42a2-981c-79dcfbe213d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:24.420928 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae785b-6876-42a2-981c-79dcfbe213d4 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:24.422114 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ae785b-6876-42a2-981c-79dcfbe213d4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:24.422114 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ae785b-6876-42a2-981c-79dcfbe213d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:24.423139 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ae785b-6876-42a2-981c-79dcfbe213d4 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:24.430824 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db5c0bc-e13c-413d-a7ec-b602d7c518b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:24.431465 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db5c0bc-e13c-413d-a7ec-b602d7c518b1 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:24.431957 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 301/582] 10.222.0.43 () {56 vars in 1103 bytes} [Tue Sep 6 16:03:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:24.432604 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ae785b-6876-42a2-981c-79dcfbe213d4 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:24.436158 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db5c0bc-e13c-413d-a7ec-b602d7c518b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:24.436458 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db5c0bc-e13c-413d-a7ec-b602d7c518b1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:24.444663 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db5c0bc-e13c-413d-a7ec-b602d7c518b1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:24.446191 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db5c0bc-e13c-413d-a7ec-b602d7c518b1 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:24.457940 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-6db5c0bc-e13c-413d-a7ec-b602d7c518b1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 16:03:24.457940 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:24.457940 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:24.457940 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:24.457940 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:24.457940 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:24.457940 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:24.457940 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:24.457940 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:24.457940 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:24.457940 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:24.457940 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:24.457940 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:03:24.457940 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:03:24.457940 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:03:24.457940 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:03:24.458860 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:24.458860 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:24.458860 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:03:24.458860 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:03:24.458860 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:03:24.458860 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:03:24.458860 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:24.458860 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:24.458860 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:24.458860 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:24.458860 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:24.458860 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:24.458860 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:24.458860 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:24.458860 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:24.458860 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:24.458860 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:24.458860 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:24.459792 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:24.459792 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:24.459792 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:03:24.459792 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:03:24.459792 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:03:24.459792 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:03:24.459792 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:03:24.459792 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:03:24.459792 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 16:03:24.459792 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:24.459792 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 282/583] 10.222.0.43 () {56 vars in 1093 bytes} [Tue Sep 6 16:03:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:24.459792 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db5c0bc-e13c-413d-a7ec-b602d7c518b1 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:24.542670 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ae785b-6876-42a2-981c-79dcfbe213d4 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:24.566965 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db5c0bc-e13c-413d-a7ec-b602d7c518b1 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:24.577381 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea28fff-36fd-471b-a21c-1c1279d2ccbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:24.577999 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea28fff-36fd-471b-a21c-1c1279d2ccbf None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:24.578275 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea28fff-36fd-471b-a21c-1c1279d2ccbf None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:24.578544 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea28fff-36fd-471b-a21c-1c1279d2ccbf None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:24.579233 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea28fff-36fd-471b-a21c-1c1279d2ccbf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:24.579535 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea28fff-36fd-471b-a21c-1c1279d2ccbf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:24.581375 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea28fff-36fd-471b-a21c-1c1279d2ccbf None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:24.593537 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 302/584] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 6 16:03:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:24.594214 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea28fff-36fd-471b-a21c-1c1279d2ccbf None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:24.606218 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:24.608354 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd652e5-caa7-40ca-9e20-6cf6cbadde31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:24.608985 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd652e5-caa7-40ca-9e20-6cf6cbadde31 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:24.609247 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd652e5-caa7-40ca-9e20-6cf6cbadde31 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:24.609519 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd652e5-caa7-40ca-9e20-6cf6cbadde31 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:24.617497 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd652e5-caa7-40ca-9e20-6cf6cbadde31 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:24.619012 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd652e5-caa7-40ca-9e20-6cf6cbadde31 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:24.631074 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-3bd652e5-caa7-40ca-9e20-6cf6cbadde31 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 06 16:03:24.631074 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:24.631074 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:24.631074 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:24.631074 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:24.631074 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:24.631074 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:24.631074 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:24.631074 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:24.631074 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:24.631074 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:24.631074 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:24.631074 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:03:24.631074 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:03:24.631074 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:03:24.631074 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:03:24.632038 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:24.632038 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:24.632038 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:03:24.632038 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:03:24.632038 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:03:24.632038 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:03:24.632038 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:24.632038 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:24.632038 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:24.632038 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:24.632038 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:24.632038 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:24.632038 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:24.632038 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:24.632038 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:24.632038 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:24.632038 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:24.632038 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:24.637785 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:24.637785 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:24.637785 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:03:24.637785 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:03:24.637785 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:03:24.637785 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:03:24.637785 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:03:24.637785 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:03:24.637785 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 06 16:03:24.637785 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:24.637785 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 283/585] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 6 16:03:24 2022] GET /identity/v3/users/cinder => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:24.637785 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd652e5-caa7-40ca-9e20-6cf6cbadde31 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:24.649643 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:24.707488 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea28fff-36fd-471b-a21c-1c1279d2ccbf None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:24.738453 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd652e5-caa7-40ca-9e20-6cf6cbadde31 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:24.741952 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bf00ce-9dd0-4802-8aef-4d6e3029cfcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:24.742567 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bf00ce-9dd0-4802-8aef-4d6e3029cfcd None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:24.742835 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bf00ce-9dd0-4802-8aef-4d6e3029cfcd None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:24.743105 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bf00ce-9dd0-4802-8aef-4d6e3029cfcd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:24.754785 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bf00ce-9dd0-4802-8aef-4d6e3029cfcd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:24.756258 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bf00ce-9dd0-4802-8aef-4d6e3029cfcd None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:24.770840 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-c3bf00ce-9dd0-4802-8aef-4d6e3029cfcd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:24.770840 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:24.770840 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:24.770840 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:24.770840 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:24.770840 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:24.770840 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:24.770840 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:24.770840 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:24.770840 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:24.770840 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:24.770840 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:24.770840 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:03:24.770840 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:03:24.770840 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:03:24.770840 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:03:24.771792 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:24.771792 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:24.771792 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:24.771792 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:24.771792 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:24.771792 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:24.771792 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:24.771792 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:24.771792 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:24.771792 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:24.771792 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:24.771792 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:24.771792 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:24.771792 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:24.771792 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:24.771792 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:24.771792 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:03:24.772670 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:03:24.772670 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:24.772670 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:24.772670 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 303/586] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:03:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:24.772919 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bf00ce-9dd0-4802-8aef-4d6e3029cfcd None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:24.784356 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afc5457-e179-4488-be0c-42a4a7f2f079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:24.785100 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afc5457-e179-4488-be0c-42a4a7f2f079 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:24.785422 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afc5457-e179-4488-be0c-42a4a7f2f079 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:24.785739 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afc5457-e179-4488-be0c-42a4a7f2f079 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:24.786485 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afc5457-e179-4488-be0c-42a4a7f2f079 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:24.786789 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afc5457-e179-4488-be0c-42a4a7f2f079 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:24.788300 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afc5457-e179-4488-be0c-42a4a7f2f079 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:24.808941 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 284/587] 10.222.0.43 () {56 vars in 1133 bytes} [Tue Sep 6 16:03:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:24.809922 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afc5457-e179-4488-be0c-42a4a7f2f079 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:24.846341 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:24.874184 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:24.890535 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bf00ce-9dd0-4802-8aef-4d6e3029cfcd None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:24.909496 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afc5457-e179-4488-be0c-42a4a7f2f079 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:24.924228 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4807d57e-cd2e-490b-a7d8-4a53a73fe009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:24.924939 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4807d57e-cd2e-490b-a7d8-4a53a73fe009 None admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:24.925194 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4807d57e-cd2e-490b-a7d8-4a53a73fe009 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:24.925463 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4807d57e-cd2e-490b-a7d8-4a53a73fe009 None admin] PATH_INFO: `/v3/projects/128d210e8dad48ac921ed4cb1d444ceb/users/2c163b39e61648c385f065ce8091c3ab/roles/31321627b02c4ff188f8fe1521ce3181` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:24.939478 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4807d57e-cd2e-490b-a7d8-4a53a73fe009 None admin] RBAC: Authorizing `identity:create_grant(project_id=128d210e8dad48ac921ed4cb1d444ceb, user_id=2c163b39e61648c385f065ce8091c3ab, role_id=31321627b02c4ff188f8fe1521ce3181)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:24.940948 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4807d57e-cd2e-490b-a7d8-4a53a73fe009 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:24.947342 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717fcd62-8141-42e3-9759-24c3cc682664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:24.948068 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717fcd62-8141-42e3-9759-24c3cc682664 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:24.948352 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717fcd62-8141-42e3-9759-24c3cc682664 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:24.948666 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717fcd62-8141-42e3-9759-24c3cc682664 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:24.949436 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717fcd62-8141-42e3-9759-24c3cc682664 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:24.949767 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717fcd62-8141-42e3-9759-24c3cc682664 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:24.951396 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717fcd62-8141-42e3-9759-24c3cc682664 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:24.964871 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 304/588] 10.222.0.43 () {58 vars in 1439 bytes} [Tue Sep 6 16:03:24 2022] PUT /identity/v3/projects/128d210e8dad48ac921ed4cb1d444ceb/users/2c163b39e61648c385f065ce8091c3ab/roles/31321627b02c4ff188f8fe1521ce3181 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:03:24.964871 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4807d57e-cd2e-490b-a7d8-4a53a73fe009 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:24.985248 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 285/589] 10.222.0.43 () {56 vars in 1137 bytes} [Tue Sep 6 16:03:24 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:24.986416 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717fcd62-8141-42e3-9759-24c3cc682664 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:25.088833 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4807d57e-cd2e-490b-a7d8-4a53a73fe009 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:25.111822 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:25.135058 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717fcd62-8141-42e3-9759-24c3cc682664 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:25.144153 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b85de5c-7077-48ac-94fd-4ac2396c813c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:25.144976 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b85de5c-7077-48ac-94fd-4ac2396c813c None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:25.145186 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b85de5c-7077-48ac-94fd-4ac2396c813c None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:25.145421 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b85de5c-7077-48ac-94fd-4ac2396c813c None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:25.146079 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b85de5c-7077-48ac-94fd-4ac2396c813c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:25.146424 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b85de5c-7077-48ac-94fd-4ac2396c813c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:25.147912 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b85de5c-7077-48ac-94fd-4ac2396c813c None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:25.162375 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 305/590] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:03:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:25.163222 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b85de5c-7077-48ac-94fd-4ac2396c813c None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:25.171846 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:25.182535 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81a1fb0-c204-4a81-8581-de9e5879ba3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:25.183402 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a1fb0-c204-4a81-8581-de9e5879ba3a None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:25.183861 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a1fb0-c204-4a81-8581-de9e5879ba3a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:25.184298 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a1fb0-c204-4a81-8581-de9e5879ba3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:25.196251 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81a1fb0-c204-4a81-8581-de9e5879ba3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:25.198228 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81a1fb0-c204-4a81-8581-de9e5879ba3a None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:25.215518 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-b81a1fb0-c204-4a81-8581-de9e5879ba3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:25.215518 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:25.215518 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:25.215518 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:25.215518 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:25.215518 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:25.215518 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:25.215518 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:25.215518 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:25.215518 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:25.215518 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:25.215518 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:25.215518 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:03:25.215518 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:03:25.215518 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:03:25.215518 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:03:25.216676 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:25.216676 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:25.216676 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:25.216676 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:25.216676 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:25.216676 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:25.216676 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:25.216676 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:25.216676 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:25.216676 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:25.216676 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:25.216676 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:25.216676 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:25.216676 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:25.216676 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:25.216676 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:25.216676 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:03:25.217749 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:03:25.217749 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:25.217749 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:25.219430 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 286/591] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:03:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:25.220347 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81a1fb0-c204-4a81-8581-de9e5879ba3a None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:25.265911 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b85de5c-7077-48ac-94fd-4ac2396c813c None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:25.299844 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b376a794-de6d-40dc-84e9-3da2fd4c20c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:25.300480 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b376a794-de6d-40dc-84e9-3da2fd4c20c0 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:25.300798 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b376a794-de6d-40dc-84e9-3da2fd4c20c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:25.301109 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b376a794-de6d-40dc-84e9-3da2fd4c20c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:25.312088 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b376a794-de6d-40dc-84e9-3da2fd4c20c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:25.313529 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b376a794-de6d-40dc-84e9-3da2fd4c20c0 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:25.328160 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-b376a794-de6d-40dc-84e9-3da2fd4c20c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:25.328160 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:25.328160 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:25.328160 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:25.328160 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:25.328160 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:25.328160 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:25.328160 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:25.328160 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:25.328160 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:25.328160 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:25.328160 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:25.328160 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:03:25.328160 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:03:25.328160 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:03:25.328160 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:03:25.329364 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:25.329364 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:25.329364 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:25.329364 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:25.329364 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:25.329364 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:25.329364 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:25.329364 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:25.329364 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:25.329364 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:25.329364 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:25.329364 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:25.329364 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:25.329364 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:25.329364 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:25.329364 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:25.329364 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:03:25.330738 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:03:25.330738 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:25.330738 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:25.330738 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 306/592] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:03:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:25.330738 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b376a794-de6d-40dc-84e9-3da2fd4c20c0 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:25.345413 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81a1fb0-c204-4a81-8581-de9e5879ba3a None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:25.362694 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:25.386783 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6a3724-66a3-4002-b90e-3b3008c5357a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:25.387741 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6a3724-66a3-4002-b90e-3b3008c5357a None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:25.388178 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6a3724-66a3-4002-b90e-3b3008c5357a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:25.388603 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6a3724-66a3-4002-b90e-3b3008c5357a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:25.397241 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6a3724-66a3-4002-b90e-3b3008c5357a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:25.399098 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6a3724-66a3-4002-b90e-3b3008c5357a None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:25.411385 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-9d6a3724-66a3-4002-b90e-3b3008c5357a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 06 16:03:25.411385 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:25.411385 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:25.411385 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:25.411385 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:25.411385 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:25.411385 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:25.411385 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:25.411385 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:25.411385 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:25.411385 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:25.411385 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:25.411385 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:03:25.411385 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:03:25.411385 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:03:25.411385 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:03:25.412573 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:25.412573 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:25.412573 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:03:25.412573 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:03:25.412573 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:03:25.412573 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:03:25.412573 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:25.412573 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:25.412573 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:25.412573 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:25.412573 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:25.412573 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:25.412573 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:25.412573 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:25.412573 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:25.412573 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:25.412573 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:25.412573 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:25.413607 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:25.413607 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:25.413607 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:03:25.413607 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:03:25.413607 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:03:25.413607 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:03:25.413607 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:03:25.413607 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:03:25.413607 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 06 16:03:25.413607 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:25.415871 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 287/593] 10.222.0.43 () {56 vars in 1102 bytes} [Tue Sep 6 16:03:25 2022] GET /identity/v3/users/neutron => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:25.417013 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6a3724-66a3-4002-b90e-3b3008c5357a None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:25.442647 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b376a794-de6d-40dc-84e9-3da2fd4c20c0 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:25.464945 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:25.477671 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbdc276-989e-467e-9621-0d01b1e0acd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:25.478215 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbdc276-989e-467e-9621-0d01b1e0acd4 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:25.478486 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbdc276-989e-467e-9621-0d01b1e0acd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:25.478771 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbdc276-989e-467e-9621-0d01b1e0acd4 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:25.479401 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbdc276-989e-467e-9621-0d01b1e0acd4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:25.479686 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbdc276-989e-467e-9621-0d01b1e0acd4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:25.481076 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbdc276-989e-467e-9621-0d01b1e0acd4 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:25.493826 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 307/594] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:03:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:25.494611 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbdc276-989e-467e-9621-0d01b1e0acd4 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:25.534488 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6a3724-66a3-4002-b90e-3b3008c5357a None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:25.571708 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305a00c9-cba5-4741-9119-65dca609ea77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:25.572631 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305a00c9-cba5-4741-9119-65dca609ea77 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:25.573171 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305a00c9-cba5-4741-9119-65dca609ea77 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:25.573578 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305a00c9-cba5-4741-9119-65dca609ea77 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:25.574459 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305a00c9-cba5-4741-9119-65dca609ea77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:25.574893 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305a00c9-cba5-4741-9119-65dca609ea77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:25.576494 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:25.577670 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305a00c9-cba5-4741-9119-65dca609ea77 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:25.592723 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbdc276-989e-467e-9621-0d01b1e0acd4 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:25.592961 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 288/595] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:03:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:25.593947 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305a00c9-cba5-4741-9119-65dca609ea77 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:25.631356 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49c85b2-33a9-4951-aa12-5034a146ea99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:25.631954 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c85b2-33a9-4951-aa12-5034a146ea99 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:25.632234 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c85b2-33a9-4951-aa12-5034a146ea99 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:25.633162 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c85b2-33a9-4951-aa12-5034a146ea99 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:25.633279 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49c85b2-33a9-4951-aa12-5034a146ea99 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:25.633525 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49c85b2-33a9-4951-aa12-5034a146ea99 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:25.634978 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49c85b2-33a9-4951-aa12-5034a146ea99 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:25.652538 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 308/596] 10.222.0.43 () {56 vars in 1139 bytes} [Tue Sep 6 16:03:25 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:25.653422 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49c85b2-33a9-4951-aa12-5034a146ea99 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:25.702592 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:25.708353 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305a00c9-cba5-4741-9119-65dca609ea77 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:25.746520 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc71798-8c30-460d-b579-86a6f8282ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:25.747504 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc71798-8c30-460d-b579-86a6f8282ca1 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:25.747944 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc71798-8c30-460d-b579-86a6f8282ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:25.748477 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc71798-8c30-460d-b579-86a6f8282ca1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:25.751952 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49c85b2-33a9-4951-aa12-5034a146ea99 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:25.756724 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc71798-8c30-460d-b579-86a6f8282ca1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:25.758489 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc71798-8c30-460d-b579-86a6f8282ca1 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:25.774504 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-efc71798-8c30-460d-b579-86a6f8282ca1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:03:25.774504 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:25.774504 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:25.774504 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:25.774504 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:25.774504 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:25.774504 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:25.774504 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:25.774504 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:25.774504 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:25.774504 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:25.774504 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:25.774504 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:03:25.774504 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:03:25.774504 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:03:25.775441 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:03:25.775441 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:25.775441 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:25.775441 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:25.775441 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:25.775441 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:25.775441 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:25.775441 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:25.775441 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:25.775441 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:25.775441 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:25.775441 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:25.775441 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:25.775441 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:25.775441 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:25.775441 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:25.775441 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:25.777063 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:03:25.777063 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:03:25.777063 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:03:25.777063 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:03:25.777063 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:03:25.777063 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:03:25.777063 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:03:25.777063 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:25.779240 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 289/597] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 6 16:03:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:25.780291 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc71798-8c30-460d-b579-86a6f8282ca1 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:25.788452 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a73141d-84f1-4f1d-af46-c375c7b39588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:25.789114 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a73141d-84f1-4f1d-af46-c375c7b39588 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:25.789388 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a73141d-84f1-4f1d-af46-c375c7b39588 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:25.789655 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a73141d-84f1-4f1d-af46-c375c7b39588 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:25.796633 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a73141d-84f1-4f1d-af46-c375c7b39588 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:25.798090 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a73141d-84f1-4f1d-af46-c375c7b39588 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:25.809625 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-7a73141d-84f1-4f1d-af46-c375c7b39588 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:03:25.809625 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:25.809625 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:25.809625 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:25.809625 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:25.809625 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:25.809625 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:25.809625 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:25.809625 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:25.809625 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:25.809625 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:25.809625 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:25.809625 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:03:25.809625 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:03:25.809625 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:03:25.810671 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:03:25.810671 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:25.810671 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:25.810671 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:25.810671 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:25.810671 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:25.810671 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:25.810671 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:25.810671 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:25.810671 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:25.810671 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:25.810671 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:25.810671 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:25.810671 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:25.810671 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:25.810671 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:25.810671 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:25.811662 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:03:25.811662 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:03:25.811662 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:03:25.811662 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:03:25.811662 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:03:25.811662 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:03:25.811662 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:03:25.811662 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:25.815428 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 309/598] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 6 16:03:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:25.816296 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a73141d-84f1-4f1d-af46-c375c7b39588 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:25.823233 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:25.904786 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc71798-8c30-460d-b579-86a6f8282ca1 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:25.942731 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a73141d-84f1-4f1d-af46-c375c7b39588 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:25.950370 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87855ba2-e164-43e2-bfed-9ccaad18b86a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:25.951006 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87855ba2-e164-43e2-bfed-9ccaad18b86a None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:25.951810 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87855ba2-e164-43e2-bfed-9ccaad18b86a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:25.952091 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87855ba2-e164-43e2-bfed-9ccaad18b86a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:25.966702 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87855ba2-e164-43e2-bfed-9ccaad18b86a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:25.968541 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87855ba2-e164-43e2-bfed-9ccaad18b86a None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:25.980124 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c27101-7810-44d5-84e1-e245d7818d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:25.981002 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c27101-7810-44d5-84e1-e245d7818d8c None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:25.981415 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c27101-7810-44d5-84e1-e245d7818d8c None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:25.981844 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c27101-7810-44d5-84e1-e245d7818d8c None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:25.982684 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c27101-7810-44d5-84e1-e245d7818d8c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:25.983125 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c27101-7810-44d5-84e1-e245d7818d8c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:25.984673 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c27101-7810-44d5-84e1-e245d7818d8c None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:25.985543 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-87855ba2-e164-43e2-bfed-9ccaad18b86a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:25.985543 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:25.985543 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:25.985543 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:25.985543 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:25.985543 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:25.985543 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:25.985543 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:25.985543 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:25.985543 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:25.985543 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:25.985543 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:25.985543 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:03:25.985543 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:03:25.985543 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:03:25.985543 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:03:25.986605 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:25.986605 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:25.986605 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:25.986605 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:25.986605 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:25.986605 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:25.986605 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:25.986605 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:25.986605 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:25.986605 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:25.986605 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:25.986605 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:25.986605 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:25.986605 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:25.986605 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:25.986605 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:25.986605 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:03:25.987488 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:03:25.987488 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:25.987488 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:25.987488 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 290/599] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:03:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:25.987488 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87855ba2-e164-43e2-bfed-9ccaad18b86a None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:26.000524 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 310/600] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 6 16:03:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:26.001647 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c27101-7810-44d5-84e1-e245d7818d8c None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:26.002281 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:26.066626 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:26.116854 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c27101-7810-44d5-84e1-e245d7818d8c None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:26.118641 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87855ba2-e164-43e2-bfed-9ccaad18b86a None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:26.161480 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed005108-11e6-4298-9a2a-97be44e7d017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:26.162065 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed005108-11e6-4298-9a2a-97be44e7d017 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:26.162336 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed005108-11e6-4298-9a2a-97be44e7d017 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:26.162597 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed005108-11e6-4298-9a2a-97be44e7d017 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:26.163240 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed005108-11e6-4298-9a2a-97be44e7d017 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:26.163544 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed005108-11e6-4298-9a2a-97be44e7d017 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:26.164390 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70664b4-0c90-4034-99e7-3675a76e8ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:26.164920 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed005108-11e6-4298-9a2a-97be44e7d017 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:26.165671 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70664b4-0c90-4034-99e7-3675a76e8ec0 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:26.166079 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70664b4-0c90-4034-99e7-3675a76e8ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:26.166639 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70664b4-0c90-4034-99e7-3675a76e8ec0 None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:26.167930 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70664b4-0c90-4034-99e7-3675a76e8ec0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:26.168388 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70664b4-0c90-4034-99e7-3675a76e8ec0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:26.170210 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70664b4-0c90-4034-99e7-3675a76e8ec0 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:26.180819 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 311/601] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:03:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:26.180819 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed005108-11e6-4298-9a2a-97be44e7d017 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:26.184734 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 291/602] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 6 16:03:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:26.187686 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1953f9db-50a9-43f9-9bdf-510be403e4c3 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:26.187686 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1953f9db-50a9-43f9-9bdf-510be403e4c3 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:26.187686 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1953f9db-50a9-43f9-9bdf-510be403e4c3 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:26.191754 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 292/603] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:03:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:03:26.192470 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1953f9db-50a9-43f9-9bdf-510be403e4c3 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:26.267999 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:26.290296 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed005108-11e6-4298-9a2a-97be44e7d017 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:26.301070 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1953f9db-50a9-43f9-9bdf-510be403e4c3 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:26.312323 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:26.328144 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ffc856-72d3-4fc4-96ee-79f40f2af47e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:26.329449 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ffc856-72d3-4fc4-96ee-79f40f2af47e None admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:26.329736 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ffc856-72d3-4fc4-96ee-79f40f2af47e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:26.330510 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ffc856-72d3-4fc4-96ee-79f40f2af47e None admin] PATH_INFO: `/v3/projects/128d210e8dad48ac921ed4cb1d444ceb/users/63de41f4348243998029f7d07aa5dc40/roles/31321627b02c4ff188f8fe1521ce3181` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:26.338415 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f80912-5ee5-4642-b624-e440a7234d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:26.339023 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f80912-5ee5-4642-b624-e440a7234d6d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:26.339295 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f80912-5ee5-4642-b624-e440a7234d6d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:26.339951 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f80912-5ee5-4642-b624-e440a7234d6d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:26.345424 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ffc856-72d3-4fc4-96ee-79f40f2af47e None admin] RBAC: Authorizing `identity:create_grant(project_id=128d210e8dad48ac921ed4cb1d444ceb, user_id=63de41f4348243998029f7d07aa5dc40, role_id=31321627b02c4ff188f8fe1521ce3181)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:26.346876 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f80912-5ee5-4642-b624-e440a7234d6d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:26.346998 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ffc856-72d3-4fc4-96ee-79f40f2af47e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:26.348345 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f80912-5ee5-4642-b624-e440a7234d6d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:26.359844 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-07f80912-5ee5-4642-b624-e440a7234d6d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:03:26.359844 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:26.359844 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:26.359844 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:26.359844 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:26.359844 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:26.359844 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:26.359844 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:26.359844 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:26.359844 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:26.359844 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:26.359844 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:26.359844 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:03:26.359844 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:03:26.359844 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:03:26.361150 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:03:26.361150 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:26.361150 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:26.361150 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:26.361150 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:26.361150 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:26.361150 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:26.361150 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:26.361150 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:26.361150 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:26.361150 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:26.361150 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:26.361150 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:26.361150 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:26.361150 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:26.361150 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:26.361150 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:26.362246 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:03:26.362246 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:03:26.362246 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:03:26.362246 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:03:26.362246 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:03:26.362246 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:03:26.362246 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:03:26.362246 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:26.362756 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 293/605] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 6 16:03:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:26.363610 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f80912-5ee5-4642-b624-e440a7234d6d None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:26.369861 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 312/605] 10.222.0.43 () {58 vars in 1439 bytes} [Tue Sep 6 16:03:26 2022] PUT /identity/v3/projects/128d210e8dad48ac921ed4cb1d444ceb/users/63de41f4348243998029f7d07aa5dc40/roles/31321627b02c4ff188f8fe1521ce3181 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:03:26.370480 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb376d4c-66fa-4691-82aa-23d8bf7b5174 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:26.370762 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb376d4c-66fa-4691-82aa-23d8bf7b5174 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:26.371861 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb376d4c-66fa-4691-82aa-23d8bf7b5174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:26.404128 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-cb376d4c-66fa-4691-82aa-23d8bf7b5174 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:03:26.405549 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-cb376d4c-66fa-4691-82aa-23d8bf7b5174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:03:26.489158 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 313/606] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:03:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:03:26.490316 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb376d4c-66fa-4691-82aa-23d8bf7b5174 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:26.506839 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f80912-5ee5-4642-b624-e440a7234d6d None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:26.540298 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:26.548073 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586955b6-f58e-4250-968b-f0af65f3f176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:26.548741 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586955b6-f58e-4250-968b-f0af65f3f176 None admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:26.549055 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586955b6-f58e-4250-968b-f0af65f3f176 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:26.549323 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586955b6-f58e-4250-968b-f0af65f3f176 None admin] PATH_INFO: `/v3/projects/128d210e8dad48ac921ed4cb1d444ceb/users/d1b05748e4284153b0728164cd508dc9/roles/31321627b02c4ff188f8fe1521ce3181` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:26.565890 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586955b6-f58e-4250-968b-f0af65f3f176 None admin] RBAC: Authorizing `identity:create_grant(project_id=128d210e8dad48ac921ed4cb1d444ceb, user_id=d1b05748e4284153b0728164cd508dc9, role_id=31321627b02c4ff188f8fe1521ce3181)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:26.567392 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586955b6-f58e-4250-968b-f0af65f3f176 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:26.583990 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 294/607] 10.222.0.43 () {58 vars in 1439 bytes} [Tue Sep 6 16:03:26 2022] PUT /identity/v3/projects/128d210e8dad48ac921ed4cb1d444ceb/users/d1b05748e4284153b0728164cd508dc9/roles/31321627b02c4ff188f8fe1521ce3181 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:03:26.626879 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb376d4c-66fa-4691-82aa-23d8bf7b5174 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:26.628924 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:26.666179 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1896c8-bcb6-468c-8eb1-fe3212676d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:26.666815 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1896c8-bcb6-468c-8eb1-fe3212676d9e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:26.667089 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1896c8-bcb6-468c-8eb1-fe3212676d9e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:26.667344 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1896c8-bcb6-468c-8eb1-fe3212676d9e None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:26.668053 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1896c8-bcb6-468c-8eb1-fe3212676d9e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:26.668346 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1896c8-bcb6-468c-8eb1-fe3212676d9e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:26.669853 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1896c8-bcb6-468c-8eb1-fe3212676d9e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:26.683426 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 314/608] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 6 16:03:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:26.689420 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586955b6-f58e-4250-968b-f0af65f3f176 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:26.795245 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586955b6-f58e-4250-968b-f0af65f3f176 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:26.830251 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde4b22c-cbf8-4516-b522-59eaef329649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:26.830916 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde4b22c-cbf8-4516-b522-59eaef329649 None admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:26.831195 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde4b22c-cbf8-4516-b522-59eaef329649 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:26.831460 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde4b22c-cbf8-4516-b522-59eaef329649 None admin] PATH_INFO: `/v3/projects/128d210e8dad48ac921ed4cb1d444ceb/users/10565aefeaaa40549ed38bb9e0f4b189/roles/31321627b02c4ff188f8fe1521ce3181` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:26.846636 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde4b22c-cbf8-4516-b522-59eaef329649 None admin] RBAC: Authorizing `identity:create_grant(project_id=128d210e8dad48ac921ed4cb1d444ceb, user_id=10565aefeaaa40549ed38bb9e0f4b189, role_id=31321627b02c4ff188f8fe1521ce3181)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:26.848283 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde4b22c-cbf8-4516-b522-59eaef329649 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:26.869727 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 295/609] 10.222.0.43 () {58 vars in 1439 bytes} [Tue Sep 6 16:03:26 2022] PUT /identity/v3/projects/128d210e8dad48ac921ed4cb1d444ceb/users/10565aefeaaa40549ed38bb9e0f4b189/roles/31321627b02c4ff188f8fe1521ce3181 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:03:26.889480 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1896c8-bcb6-468c-8eb1-fe3212676d9e None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:27.003785 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1896c8-bcb6-468c-8eb1-fe3212676d9e None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:27.006768 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb75ba4-0ffa-440b-909c-6a844a54ce38 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:27.007040 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb75ba4-0ffa-440b-909c-6a844a54ce38 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:27.007304 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb75ba4-0ffa-440b-909c-6a844a54ce38 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:27.008653 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 296/610] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:03:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:03:27.014254 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd7f35a-cca6-4304-8fb8-275d99e00c2b None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:27.014529 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd7f35a-cca6-4304-8fb8-275d99e00c2b None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:27.014806 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd7f35a-cca6-4304-8fb8-275d99e00c2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:27.038733 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-5fd7f35a-cca6-4304-8fb8-275d99e00c2b None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:03:27.039977 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5fd7f35a-cca6-4304-8fb8-275d99e00c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:03:27.054012 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a889eae0-cdb9-413d-90cd-07b6e9834aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:27.054571 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a889eae0-cdb9-413d-90cd-07b6e9834aba None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:27.054846 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a889eae0-cdb9-413d-90cd-07b6e9834aba None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:27.055207 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a889eae0-cdb9-413d-90cd-07b6e9834aba None admin] PATH_INFO: `/v3/roles/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:27.078323 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a889eae0-cdb9-413d-90cd-07b6e9834aba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:03:27.087706 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-a889eae0-cdb9-413d-90cd-07b6e9834aba None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:03:27.087706 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:27.087706 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:27.087706 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:27.087706 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:27.087706 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:27.087706 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:27.087706 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:27.087706 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:27.087706 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:27.087706 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:27.087706 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:27.087706 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:03:27.087706 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:03:27.087706 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:03:27.087706 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:03:27.088696 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:03:27.088696 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:03:27.088696 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:03:27.088696 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:03:27.088696 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:27.088696 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:27.088696 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:27.088696 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:27.088696 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:27.088696 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:27.088696 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:27.088696 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:27.088696 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:27.088696 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:27.088696 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:27.088696 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:27.089576 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:27.089576 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:27.089576 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:27.089576 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:27.089576 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:03:27.089576 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:03:27.089576 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:03:27.089576 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:03:27.089576 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:03:27.089576 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:03:27.089576 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:03:27.089576 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:27.089576 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 315/611] 10.222.0.43 () {56 vars in 1102 bytes} [Tue Sep 6 16:03:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:27.096637 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a889eae0-cdb9-413d-90cd-07b6e9834aba None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:27.098658 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:27.129469 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:27.164861 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 297/612] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:03:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 16:03:27.216648 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a889eae0-cdb9-413d-90cd-07b6e9834aba None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:27.254261 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d034a92-5e97-48c6-9505-e4204372befc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:27.254882 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d034a92-5e97-48c6-9505-e4204372befc None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:27.255205 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d034a92-5e97-48c6-9505-e4204372befc None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:27.255474 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d034a92-5e97-48c6-9505-e4204372befc None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:27.256143 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d034a92-5e97-48c6-9505-e4204372befc None admin] RBAC: Adding query filter params (name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:27.256434 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d034a92-5e97-48c6-9505-e4204372befc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:27.257828 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d034a92-5e97-48c6-9505-e4204372befc None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:27.266656 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 316/613] 10.222.0.43 () {56 vars in 1103 bytes} [Tue Sep 6 16:03:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:27.273897 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd7f35a-cca6-4304-8fb8-275d99e00c2b None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:27.418574 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd7f35a-cca6-4304-8fb8-275d99e00c2b None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:27.453030 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c62700-62d4-44bf-a510-43aa4ec83925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:27.453911 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c62700-62d4-44bf-a510-43aa4ec83925 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:27.454342 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c62700-62d4-44bf-a510-43aa4ec83925 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:27.455113 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c62700-62d4-44bf-a510-43aa4ec83925 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:27.465499 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c62700-62d4-44bf-a510-43aa4ec83925 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:27.467232 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c62700-62d4-44bf-a510-43aa4ec83925 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:27.482366 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-c4c62700-62d4-44bf-a510-43aa4ec83925 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:27.482366 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:27.482366 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:27.482366 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:27.482366 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:27.482366 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:27.482366 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:27.482366 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:27.482366 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:27.482366 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:27.482366 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:27.482366 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:27.482366 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:03:27.482366 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:03:27.482366 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:03:27.482366 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:03:27.483520 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:27.483520 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:27.483520 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:27.483520 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:27.483520 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:27.483520 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:27.483520 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:27.483520 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:27.483520 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:27.483520 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:27.483520 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:27.483520 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:27.483520 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:27.483520 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:27.483520 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:27.483520 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:27.483520 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:03:27.484600 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:03:27.484600 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:27.484600 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:27.486270 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 298/614] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:03:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:27.492029 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d034a92-5e97-48c6-9505-e4204372befc None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:27.527304 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c62700-62d4-44bf-a510-43aa4ec83925 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:27.550516 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:27.608696 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:27.628152 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d034a92-5e97-48c6-9505-e4204372befc None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:27.666847 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b5779e-8e2e-4615-898f-712b12965028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:27.667757 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b5779e-8e2e-4615-898f-712b12965028 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:27.668190 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b5779e-8e2e-4615-898f-712b12965028 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:27.668670 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b5779e-8e2e-4615-898f-712b12965028 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:27.669608 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b5779e-8e2e-4615-898f-712b12965028 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:27.670042 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b5779e-8e2e-4615-898f-712b12965028 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:27.671795 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b5779e-8e2e-4615-898f-712b12965028 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:27.692122 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 317/615] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:03:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:27.697832 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b5779e-8e2e-4615-898f-712b12965028 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:27.748709 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c62700-62d4-44bf-a510-43aa4ec83925 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:27.789993 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d8e656-118d-426f-bf94-f2e33304b15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:27.790870 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d8e656-118d-426f-bf94-f2e33304b15e None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:27.791291 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d8e656-118d-426f-bf94-f2e33304b15e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:27.791720 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d8e656-118d-426f-bf94-f2e33304b15e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:27.802933 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d8e656-118d-426f-bf94-f2e33304b15e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:03:27.814520 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-12d8e656-118d-426f-bf94-f2e33304b15e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:03:27.814520 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:27.814520 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:27.814520 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:27.814520 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:27.814520 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:27.814520 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:27.814520 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:27.814520 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:27.814520 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:27.814520 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:27.814520 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:27.814520 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:03:27.814520 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:03:27.814520 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:03:27.814520 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:03:27.815568 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:03:27.815568 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:03:27.815568 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:03:27.815568 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:03:27.815568 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:27.815568 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:27.815568 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:27.815568 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:27.815568 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:27.815568 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:27.815568 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:27.815568 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:27.815568 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:27.815568 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:27.815568 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:27.815568 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:27.816473 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:27.816473 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:27.816473 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:27.816473 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:27.816473 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:03:27.816473 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:03:27.816473 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:03:27.816473 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:03:27.816473 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:03:27.816473 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:03:27.816473 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:03:27.816473 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:27.818817 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 299/616] 10.222.0.43 () {56 vars in 1102 bytes} [Tue Sep 6 16:03:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:03:27.824201 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d8e656-118d-426f-bf94-f2e33304b15e None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:27.852821 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b5779e-8e2e-4615-898f-712b12965028 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:27.905458 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac61299-d57c-4640-9ecd-7d1f2a98fc6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:27.906114 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac61299-d57c-4640-9ecd-7d1f2a98fc6c None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:27.906533 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac61299-d57c-4640-9ecd-7d1f2a98fc6c None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:27.906672 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac61299-d57c-4640-9ecd-7d1f2a98fc6c None admin] PATH_INFO: `/v3/users/swift` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:27.919252 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:27.933536 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac61299-d57c-4640-9ecd-7d1f2a98fc6c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:27.935141 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac61299-d57c-4640-9ecd-7d1f2a98fc6c None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:27.952999 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d8e656-118d-426f-bf94-f2e33304b15e None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:27.958784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-dac61299-d57c-4640-9ecd-7d1f2a98fc6c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 06 16:03:27.958784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:27.958784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:27.958784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:27.958784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:27.958784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:27.958784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:27.958784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:27.958784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:27.958784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:27.958784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:27.958784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:27.958784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:03:27.958784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:03:27.958784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:03:27.958784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:03:27.959868 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:27.959868 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:27.959868 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:03:27.959868 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:03:27.959868 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:03:27.959868 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:03:27.959868 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:27.959868 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:27.959868 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:27.959868 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:27.959868 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:27.959868 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:27.959868 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:27.959868 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:27.959868 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:27.959868 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:27.959868 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:27.959868 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:27.961068 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:27.961068 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:27.961068 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:03:27.961068 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:03:27.961068 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:03:27.961068 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:03:27.961068 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:03:27.961068 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:03:27.961068 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 06 16:03:27.961068 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:27.969187 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 318/617] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 6 16:03:27 2022] GET /identity/v3/users/swift => generated 83 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:03:27.974871 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac61299-d57c-4640-9ecd-7d1f2a98fc6c None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:27.984157 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:27.994748 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9865f655-60cb-4606-9ee8-aa1bedba821d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:27.995640 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9865f655-60cb-4606-9ee8-aa1bedba821d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:27.996083 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9865f655-60cb-4606-9ee8-aa1bedba821d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:27.996491 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9865f655-60cb-4606-9ee8-aa1bedba821d None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:27.997324 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9865f655-60cb-4606-9ee8-aa1bedba821d None admin] RBAC: Adding query filter params (name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:27.997758 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9865f655-60cb-4606-9ee8-aa1bedba821d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:27.999389 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9865f655-60cb-4606-9ee8-aa1bedba821d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:28.009199 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 300/618] 10.222.0.43 () {56 vars in 1103 bytes} [Tue Sep 6 16:03:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:03:28.015095 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9865f655-60cb-4606-9ee8-aa1bedba821d None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:28.108623 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac61299-d57c-4640-9ecd-7d1f2a98fc6c None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:28.143966 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4575edde-a36e-43a8-b2e0-9b51f5e8ca1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:28.144850 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4575edde-a36e-43a8-b2e0-9b51f5e8ca1f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:28.145230 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4575edde-a36e-43a8-b2e0-9b51f5e8ca1f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:28.145608 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4575edde-a36e-43a8-b2e0-9b51f5e8ca1f None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:28.146475 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4575edde-a36e-43a8-b2e0-9b51f5e8ca1f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:28.146852 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4575edde-a36e-43a8-b2e0-9b51f5e8ca1f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:28.148462 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4575edde-a36e-43a8-b2e0-9b51f5e8ca1f None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:28.156647 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9865f655-60cb-4606-9ee8-aa1bedba821d None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:28.183271 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 319/619] 10.222.0.43 () {56 vars in 1135 bytes} [Tue Sep 6 16:03:27 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:03:28.188794 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4575edde-a36e-43a8-b2e0-9b51f5e8ca1f None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:28.192817 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3ecad1-ff15-460a-9046-79bc1f3f3548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:28.194036 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ecad1-ff15-460a-9046-79bc1f3f3548 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:28.194462 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ecad1-ff15-460a-9046-79bc1f3f3548 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:28.194882 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ecad1-ff15-460a-9046-79bc1f3f3548 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:28.197289 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:28.209903 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3ecad1-ff15-460a-9046-79bc1f3f3548 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:28.211901 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3ecad1-ff15-460a-9046-79bc1f3f3548 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:28.250206 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-1f3ecad1-ff15-460a-9046-79bc1f3f3548 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:28.250206 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:28.250206 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:28.250206 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:28.250206 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:28.250206 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:28.250206 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:28.250206 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:28.250206 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:28.250206 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:28.250206 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:28.250206 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:28.250206 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:03:28.250206 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:03:28.250206 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:03:28.250206 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:03:28.251374 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:28.251374 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:28.251374 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:28.251374 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:28.251374 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:28.251374 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:28.251374 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:28.251374 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:28.251374 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:28.251374 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:28.251374 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:28.251374 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:28.251374 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:28.251374 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:28.251374 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:28.251374 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:28.251374 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:03:28.252453 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:03:28.252453 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:28.252453 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:28.258187 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 301/620] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:03:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:03:28.263956 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3ecad1-ff15-460a-9046-79bc1f3f3548 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:28.265683 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:28.326531 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4575edde-a36e-43a8-b2e0-9b51f5e8ca1f None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:28.369444 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78481622-7cfe-443f-be01-7fb6edfb26cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:28.370455 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78481622-7cfe-443f-be01-7fb6edfb26cd None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:28.370841 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78481622-7cfe-443f-be01-7fb6edfb26cd None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:28.371420 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78481622-7cfe-443f-be01-7fb6edfb26cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:28.384706 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78481622-7cfe-443f-be01-7fb6edfb26cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:28.384706 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78481622-7cfe-443f-be01-7fb6edfb26cd None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:28.400934 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-78481622-7cfe-443f-be01-7fb6edfb26cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:28.400934 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:28.400934 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:28.400934 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:28.400934 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:28.400934 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:28.400934 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:28.400934 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:28.400934 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:28.400934 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:28.400934 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:28.400934 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:28.400934 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:03:28.400934 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:03:28.400934 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:03:28.400934 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:03:28.401904 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:28.401904 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:28.401904 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:28.401904 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:28.401904 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:28.401904 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:28.401904 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:28.401904 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:28.401904 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:28.401904 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:28.401904 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:28.401904 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:28.401904 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:28.401904 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:28.401904 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:28.401904 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:28.401904 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:03:28.402782 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:03:28.402782 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:28.402782 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:28.405335 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 320/621] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:03:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:28.406549 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78481622-7cfe-443f-be01-7fb6edfb26cd None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:28.423569 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3ecad1-ff15-460a-9046-79bc1f3f3548 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:28.465868 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b30cbf-2956-4f65-8c6c-8ea1b2172a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:28.466404 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b30cbf-2956-4f65-8c6c-8ea1b2172a9d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:28.466681 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b30cbf-2956-4f65-8c6c-8ea1b2172a9d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:28.466949 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b30cbf-2956-4f65-8c6c-8ea1b2172a9d None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:28.467600 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b30cbf-2956-4f65-8c6c-8ea1b2172a9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:28.467902 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b30cbf-2956-4f65-8c6c-8ea1b2172a9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:28.469603 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b30cbf-2956-4f65-8c6c-8ea1b2172a9d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:28.483880 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 302/622] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:03:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:28.489360 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b30cbf-2956-4f65-8c6c-8ea1b2172a9d None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:28.575268 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:28.625228 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b30cbf-2956-4f65-8c6c-8ea1b2172a9d None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:28.650717 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78481622-7cfe-443f-be01-7fb6edfb26cd None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:28.662530 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c085be67-cd0c-408f-9577-78bf84a24672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:28.663181 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c085be67-cd0c-408f-9577-78bf84a24672 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:28.663454 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c085be67-cd0c-408f-9577-78bf84a24672 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:28.664210 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c085be67-cd0c-408f-9577-78bf84a24672 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:28.672070 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c085be67-cd0c-408f-9577-78bf84a24672 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:28.673558 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c085be67-cd0c-408f-9577-78bf84a24672 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:28.686899 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-c085be67-cd0c-408f-9577-78bf84a24672 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 06 16:03:28.686899 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:28.686899 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:28.686899 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:28.686899 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:28.686899 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:28.686899 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:28.686899 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:28.686899 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:28.686899 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:28.686899 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:28.686899 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:28.686899 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:03:28.686899 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:03:28.686899 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:03:28.686899 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:03:28.687920 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:28.687920 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:28.687920 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:03:28.687920 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:03:28.687920 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:03:28.687920 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:03:28.687920 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:28.687920 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:28.687920 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:28.687920 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:28.687920 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:28.687920 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:28.687920 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:28.687920 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:28.687920 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:28.687920 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:28.687920 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:28.687920 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:28.689397 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:28.689397 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:28.689397 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:03:28.689397 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:03:28.689397 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:03:28.689397 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:03:28.689397 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:03:28.689397 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:03:28.689397 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 06 16:03:28.689397 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:28.690079 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:28.690212 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 303/623] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 6 16:03:28 2022] GET /identity/v3/users/glance => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:28.697774 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c085be67-cd0c-408f-9577-78bf84a24672 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:28.734914 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd25ad96-08d6-47de-8269-e585be943ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:28.735436 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd25ad96-08d6-47de-8269-e585be943ac4 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:28.735706 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd25ad96-08d6-47de-8269-e585be943ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:28.735967 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd25ad96-08d6-47de-8269-e585be943ac4 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:28.736609 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd25ad96-08d6-47de-8269-e585be943ac4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:28.736939 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd25ad96-08d6-47de-8269-e585be943ac4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:28.738377 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd25ad96-08d6-47de-8269-e585be943ac4 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:28.758328 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 321/624] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:03:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:03:28.764864 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd25ad96-08d6-47de-8269-e585be943ac4 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:28.869773 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c085be67-cd0c-408f-9577-78bf84a24672 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:28.877419 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:28.915217 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e36e09-dd94-4d21-923f-14efd2ecaf03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:28.915827 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e36e09-dd94-4d21-923f-14efd2ecaf03 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:28.916117 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e36e09-dd94-4d21-923f-14efd2ecaf03 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:28.916379 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e36e09-dd94-4d21-923f-14efd2ecaf03 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:28.917411 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e36e09-dd94-4d21-923f-14efd2ecaf03 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:28.917768 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e36e09-dd94-4d21-923f-14efd2ecaf03 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:28.919161 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e36e09-dd94-4d21-923f-14efd2ecaf03 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:28.924272 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd25ad96-08d6-47de-8269-e585be943ac4 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:28.937528 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 304/625] 10.222.0.43 () {56 vars in 1137 bytes} [Tue Sep 6 16:03:28 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:28.940988 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8681727e-d194-4b99-9042-634e50606218 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:28.940988 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8681727e-d194-4b99-9042-634e50606218 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:28.940988 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8681727e-d194-4b99-9042-634e50606218 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:28.944711 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 305/626] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:03:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:03:28.945588 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8681727e-d194-4b99-9042-634e50606218 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:28.974570 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da74634-e902-4e24-a7cb-60fa56ff87ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:28.975198 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da74634-e902-4e24-a7cb-60fa56ff87ea None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:28.975473 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da74634-e902-4e24-a7cb-60fa56ff87ea None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:28.975741 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da74634-e902-4e24-a7cb-60fa56ff87ea None admin] PATH_INFO: `/v3/projects/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:28.982581 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da74634-e902-4e24-a7cb-60fa56ff87ea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:28.984017 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da74634-e902-4e24-a7cb-60fa56ff87ea None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:28.995187 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-6da74634-e902-4e24-a7cb-60fa56ff87ea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:03:28.995187 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:28.995187 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:28.995187 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:28.995187 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:28.995187 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:28.995187 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:28.995187 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:28.995187 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:28.995187 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:28.995187 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:28.995187 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:28.995187 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:03:28.995187 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:03:28.995187 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:03:28.997494 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:03:28.997494 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:28.997494 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:28.997494 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:28.997494 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:28.997494 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:28.997494 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:28.997494 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:28.997494 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:28.997494 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:28.997494 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:28.997494 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:28.997494 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:28.997494 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:28.997494 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:28.997494 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:28.997494 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:28.999703 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:03:28.999703 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:03:28.999703 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:03:28.999703 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:03:28.999703 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:03:28.999703 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:03:28.999703 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:03:28.999703 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:28.999703 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 322/627] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 6 16:03:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:28.999703 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2b2b9a-1684-408f-a3ac-5b718a8d7753 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:28.999703 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2b2b9a-1684-408f-a3ac-5b718a8d7753 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:29.001224 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2b2b9a-1684-408f-a3ac-5b718a8d7753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:29.016038 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-8f2b2b9a-1684-408f-a3ac-5b718a8d7753 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:03:29.017242 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8f2b2b9a-1684-408f-a3ac-5b718a8d7753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:03:29.045357 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:29.080239 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8681727e-d194-4b99-9042-634e50606218 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:29.088215 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 323/628] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:03:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:03:29.088925 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2b2b9a-1684-408f-a3ac-5b718a8d7753 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:29.121333 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b82424-f241-4630-bed0-d67626e74619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:29.122695 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b82424-f241-4630-bed0-d67626e74619 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:29.123413 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b82424-f241-4630-bed0-d67626e74619 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:29.123915 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b82424-f241-4630-bed0-d67626e74619 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:29.165621 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:29.208020 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2b2b9a-1684-408f-a3ac-5b718a8d7753 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:29.214901 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b82424-f241-4630-bed0-d67626e74619 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:29.216395 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b82424-f241-4630-bed0-d67626e74619 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:29.244275 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cfeeab-aadb-4a9e-9574-5c8c096b6995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:29.244913 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cfeeab-aadb-4a9e-9574-5c8c096b6995 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:29.245286 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cfeeab-aadb-4a9e-9574-5c8c096b6995 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:29.245463 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cfeeab-aadb-4a9e-9574-5c8c096b6995 None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:29.248800 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cfeeab-aadb-4a9e-9574-5c8c096b6995 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:29.248800 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cfeeab-aadb-4a9e-9574-5c8c096b6995 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:29.248800 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cfeeab-aadb-4a9e-9574-5c8c096b6995 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:29.262279 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 324/629] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 6 16:03:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:29.262986 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6323e3a-be06-451e-9864-537b74de7a00 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:29.263267 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6323e3a-be06-451e-9864-537b74de7a00 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:29.263540 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6323e3a-be06-451e-9864-537b74de7a00 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:29.268848 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 325/630] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:03:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:03:29.268848 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ae32a6-8677-485f-bc33-dcee7ffa9ee9 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:29.268848 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ae32a6-8677-485f-bc33-dcee7ffa9ee9 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:29.268848 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ae32a6-8677-485f-bc33-dcee7ffa9ee9 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:29.268848 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 326/631] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:03:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:03:29.268848 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ae32a6-8677-485f-bc33-dcee7ffa9ee9 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:29.298288 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:29.342510 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 306/632] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 6 16:03:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 16:03:29.343848 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f4d14-931b-4564-bd83-ff3c0baa73e1 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:29.344135 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f4d14-931b-4564-bd83-ff3c0baa73e1 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:29.344407 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f4d14-931b-4564-bd83-ff3c0baa73e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:29.362371 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-a87f4d14-931b-4564-bd83-ff3c0baa73e1 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:03:29.363646 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a87f4d14-931b-4564-bd83-ff3c0baa73e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:03:29.385859 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ae32a6-8677-485f-bc33-dcee7ffa9ee9 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:29.420200 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:29.421409 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1599978d-e9bc-4a62-a105-f6aaaf862324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:29.421820 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1599978d-e9bc-4a62-a105-f6aaaf862324 None admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:29.422169 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1599978d-e9bc-4a62-a105-f6aaaf862324 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:29.422364 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1599978d-e9bc-4a62-a105-f6aaaf862324 None admin] PATH_INFO: `/v3/projects/128d210e8dad48ac921ed4cb1d444ceb/users/312026c080b64aff9dd483d3591288a6/roles/31321627b02c4ff188f8fe1521ce3181` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:29.428983 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 307/633] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:03:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:03:29.430238 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f039dfb1-c274-4c45-b552-d2ff2e6beb2e None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:29.430498 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f039dfb1-c274-4c45-b552-d2ff2e6beb2e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:29.430770 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f039dfb1-c274-4c45-b552-d2ff2e6beb2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:29.437280 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1599978d-e9bc-4a62-a105-f6aaaf862324 None admin] RBAC: Authorizing `identity:create_grant(project_id=128d210e8dad48ac921ed4cb1d444ceb, user_id=312026c080b64aff9dd483d3591288a6, role_id=31321627b02c4ff188f8fe1521ce3181)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:29.438714 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1599978d-e9bc-4a62-a105-f6aaaf862324 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:29.452731 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 327/634] 10.222.0.43 () {58 vars in 1439 bytes} [Tue Sep 6 16:03:29 2022] PUT /identity/v3/projects/128d210e8dad48ac921ed4cb1d444ceb/users/312026c080b64aff9dd483d3591288a6/roles/31321627b02c4ff188f8fe1521ce3181 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:03:29.453704 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1599978d-e9bc-4a62-a105-f6aaaf862324 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:29.460468 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-f039dfb1-c274-4c45-b552-d2ff2e6beb2e None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:03:29.462075 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f039dfb1-c274-4c45-b552-d2ff2e6beb2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:03:29.556910 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 308/635] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:03:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:03:29.558190 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f039dfb1-c274-4c45-b552-d2ff2e6beb2e None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:29.560967 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:29.615882 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1599978d-e9bc-4a62-a105-f6aaaf862324 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:29.653736 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7054bfe6-0764-4a8d-bb10-852f489a4968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:29.654371 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7054bfe6-0764-4a8d-bb10-852f489a4968 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:29.654718 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7054bfe6-0764-4a8d-bb10-852f489a4968 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:29.655250 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7054bfe6-0764-4a8d-bb10-852f489a4968 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:29.666555 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7054bfe6-0764-4a8d-bb10-852f489a4968 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:29.668066 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7054bfe6-0764-4a8d-bb10-852f489a4968 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:29.684835 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f039dfb1-c274-4c45-b552-d2ff2e6beb2e None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:29.685048 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-7054bfe6-0764-4a8d-bb10-852f489a4968 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:29.685048 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:29.685048 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:29.685048 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:29.685048 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:29.685048 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:29.685048 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:29.685048 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:29.685048 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:29.685048 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:29.685048 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:29.685048 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:29.685048 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:03:29.685048 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:03:29.685048 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:03:29.685048 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:03:29.686155 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:29.686155 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:29.686155 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:29.686155 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:29.686155 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:29.686155 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:29.686155 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:29.686155 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:29.686155 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:29.686155 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:29.686155 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:29.686155 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:29.686155 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:29.686155 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:29.686155 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:29.686155 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:29.686155 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:03:29.687275 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:03:29.687275 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:29.687275 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:29.688556 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 328/636] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:03:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:29.700110 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7054bfe6-0764-4a8d-bb10-852f489a4968 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:29.721362 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:29.740250 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23a4624-7d67-4d03-9b0d-2a771ff4241c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:29.740943 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23a4624-7d67-4d03-9b0d-2a771ff4241c None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:29.741203 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23a4624-7d67-4d03-9b0d-2a771ff4241c None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:29.741477 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23a4624-7d67-4d03-9b0d-2a771ff4241c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:29.752377 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23a4624-7d67-4d03-9b0d-2a771ff4241c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:03:29.757337 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-f23a4624-7d67-4d03-9b0d-2a771ff4241c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:03:29.757337 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:29.757337 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:29.757337 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:29.757337 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:29.757337 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:29.757337 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:29.757337 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:29.757337 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:29.757337 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:29.757337 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:29.757337 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:29.757337 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:03:29.757337 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:03:29.757337 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:03:29.757337 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:03:29.758454 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:03:29.758454 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:03:29.758454 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:03:29.758454 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:03:29.758454 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:29.758454 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:29.758454 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:29.758454 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:29.758454 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:29.758454 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:29.758454 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:29.758454 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:29.758454 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:29.758454 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:29.758454 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:29.758454 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:29.759486 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:29.759486 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:29.759486 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:29.759486 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:29.759486 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:03:29.759486 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:03:29.759486 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:03:29.759486 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:03:29.759486 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:03:29.759486 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:03:29.759486 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:03:29.759486 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:29.759486 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 309/637] 10.222.0.43 () {56 vars in 1102 bytes} [Tue Sep 6 16:03:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:29.760400 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23a4624-7d67-4d03-9b0d-2a771ff4241c None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:29.842544 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7054bfe6-0764-4a8d-bb10-852f489a4968 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:29.877438 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:29.878251 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23a4624-7d67-4d03-9b0d-2a771ff4241c None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:29.887501 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33b44cb-2b65-4a98-a978-f3e9b1818d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:29.888406 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33b44cb-2b65-4a98-a978-f3e9b1818d3b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:29.888934 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33b44cb-2b65-4a98-a978-f3e9b1818d3b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:29.889460 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33b44cb-2b65-4a98-a978-f3e9b1818d3b None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:29.890310 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33b44cb-2b65-4a98-a978-f3e9b1818d3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:29.890842 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33b44cb-2b65-4a98-a978-f3e9b1818d3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:29.892385 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33b44cb-2b65-4a98-a978-f3e9b1818d3b None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:29.905362 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 329/638] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:03:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:03:29.905362 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33b44cb-2b65-4a98-a978-f3e9b1818d3b None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:29.918731 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8df3a0-4c0f-451b-8158-315b7ca188d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:29.919373 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8df3a0-4c0f-451b-8158-315b7ca188d4 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:29.919650 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8df3a0-4c0f-451b-8158-315b7ca188d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:29.919919 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8df3a0-4c0f-451b-8158-315b7ca188d4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:29.931505 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8df3a0-4c0f-451b-8158-315b7ca188d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:03:29.937670 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-6e8df3a0-4c0f-451b-8158-315b7ca188d4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:03:29.937670 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:29.937670 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:29.937670 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:29.937670 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:29.937670 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:29.937670 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:29.937670 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:29.937670 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:29.937670 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:29.937670 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:29.937670 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:29.937670 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:03:29.937670 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:03:29.937670 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:03:29.937670 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:03:29.938688 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:03:29.938688 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:03:29.938688 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:03:29.938688 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:03:29.938688 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:29.938688 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:29.938688 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:29.938688 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:29.938688 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:29.938688 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:29.938688 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:29.938688 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:29.938688 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:29.938688 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:29.938688 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:29.938688 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:29.939518 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:29.939518 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:29.939518 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:29.939518 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:29.939518 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:03:29.939518 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:03:29.939518 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:03:29.939518 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:03:29.939518 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:03:29.939518 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:03:29.939518 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:03:29.939518 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:29.939518 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 310/639] 10.222.0.43 () {56 vars in 1102 bytes} [Tue Sep 6 16:03:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:29.939518 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8df3a0-4c0f-451b-8158-315b7ca188d4 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:29.990719 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:30.025400 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33b44cb-2b65-4a98-a978-f3e9b1818d3b None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:30.048448 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8df3a0-4c0f-451b-8158-315b7ca188d4 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:30.061918 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ab0d88-7706-401c-b140-1a392f79a2a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:30.062530 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab0d88-7706-401c-b140-1a392f79a2a7 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:30.062805 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab0d88-7706-401c-b140-1a392f79a2a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:30.063070 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab0d88-7706-401c-b140-1a392f79a2a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:30.063741 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ab0d88-7706-401c-b140-1a392f79a2a7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:30.064038 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ab0d88-7706-401c-b140-1a392f79a2a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:30.065442 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ab0d88-7706-401c-b140-1a392f79a2a7 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:30.078214 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 330/640] 10.222.0.43 () {56 vars in 1103 bytes} [Tue Sep 6 16:03:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:30.078214 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ab0d88-7706-401c-b140-1a392f79a2a7 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:30.093334 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0300ea02-acb8-455b-b4c6-6040774d0cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:30.094299 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0300ea02-acb8-455b-b4c6-6040774d0cdc None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:30.094592 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0300ea02-acb8-455b-b4c6-6040774d0cdc None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:30.094832 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0300ea02-acb8-455b-b4c6-6040774d0cdc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:30.110315 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0300ea02-acb8-455b-b4c6-6040774d0cdc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:03:30.115891 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-0300ea02-acb8-455b-b4c6-6040774d0cdc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:03:30.115891 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:30.115891 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:30.115891 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:30.115891 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:30.115891 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:30.115891 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:30.115891 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:30.115891 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:30.115891 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:30.115891 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:30.115891 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:30.115891 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:03:30.115891 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:03:30.115891 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:03:30.115891 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:03:30.116896 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:03:30.116896 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:03:30.116896 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:03:30.116896 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:03:30.116896 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:30.116896 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:30.116896 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:30.116896 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:30.116896 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:30.116896 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:30.116896 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:30.116896 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:30.116896 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:30.116896 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:30.116896 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:30.116896 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:30.117740 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:30.117740 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:30.117740 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:30.117740 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:30.117740 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:03:30.117740 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:03:30.117740 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:03:30.117740 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:03:30.117740 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:03:30.117740 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:03:30.117740 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:03:30.117740 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:30.118940 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 311/641] 10.222.0.43 () {56 vars in 1102 bytes} [Tue Sep 6 16:03:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:30.119010 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0300ea02-acb8-455b-b4c6-6040774d0cdc None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:30.129629 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:30.186341 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ab0d88-7706-401c-b140-1a392f79a2a7 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:30.221882 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5034c90-2d2c-4151-a682-310e9cfcdc9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:30.222547 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5034c90-2d2c-4151-a682-310e9cfcdc9f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:30.222813 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5034c90-2d2c-4151-a682-310e9cfcdc9f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:30.223081 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5034c90-2d2c-4151-a682-310e9cfcdc9f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:30.225831 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0300ea02-acb8-455b-b4c6-6040774d0cdc None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:30.228815 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:30.230982 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5034c90-2d2c-4151-a682-310e9cfcdc9f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:30.232468 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5034c90-2d2c-4151-a682-310e9cfcdc9f None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:30.244908 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-f5034c90-2d2c-4151-a682-310e9cfcdc9f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:03:30.244908 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:30.244908 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:30.244908 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:30.244908 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:30.244908 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:30.244908 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:30.244908 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:30.244908 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:30.244908 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:30.244908 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:30.244908 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:30.244908 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:03:30.244908 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:03:30.244908 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:03:30.245865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:03:30.245865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:30.245865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:30.245865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:30.245865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:30.245865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:30.245865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:30.245865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:30.245865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:30.245865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:30.245865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:30.245865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:30.245865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:30.245865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:30.245865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:30.245865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:30.245865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:30.252111 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:03:30.252111 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:03:30.252111 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:03:30.252111 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:03:30.252111 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:03:30.252111 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:03:30.252111 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:03:30.252111 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:30.252111 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 331/642] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 6 16:03:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:30.252111 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5034c90-2d2c-4151-a682-310e9cfcdc9f None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:30.278629 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d5b065-d33a-4e32-b795-885edc5cdb77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:30.279251 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d5b065-d33a-4e32-b795-885edc5cdb77 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:30.279527 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d5b065-d33a-4e32-b795-885edc5cdb77 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:30.279991 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d5b065-d33a-4e32-b795-885edc5cdb77 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:30.280467 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d5b065-d33a-4e32-b795-885edc5cdb77 None admin] RBAC: Adding query filter params (name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:30.280785 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d5b065-d33a-4e32-b795-885edc5cdb77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:30.282193 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d5b065-d33a-4e32-b795-885edc5cdb77 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:30.290618 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 312/643] 10.222.0.43 () {56 vars in 1103 bytes} [Tue Sep 6 16:03:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:30.295880 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d5b065-d33a-4e32-b795-885edc5cdb77 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:30.355602 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5034c90-2d2c-4151-a682-310e9cfcdc9f None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:30.364678 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:30.393091 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13505aa1-4a9e-4a78-babf-92a4b71f3d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:30.393727 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13505aa1-4a9e-4a78-babf-92a4b71f3d14 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:30.394014 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13505aa1-4a9e-4a78-babf-92a4b71f3d14 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:30.394330 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13505aa1-4a9e-4a78-babf-92a4b71f3d14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:30.403979 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d5b065-d33a-4e32-b795-885edc5cdb77 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:30.405295 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13505aa1-4a9e-4a78-babf-92a4b71f3d14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:30.406768 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13505aa1-4a9e-4a78-babf-92a4b71f3d14 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:30.421381 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-13505aa1-4a9e-4a78-babf-92a4b71f3d14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:30.421381 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:30.421381 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:30.421381 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:30.421381 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:30.421381 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:30.421381 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:30.421381 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:30.421381 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:30.421381 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:30.421381 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:30.421381 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:30.421381 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:03:30.421381 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:03:30.421381 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:03:30.421381 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:03:30.422335 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:30.422335 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:30.422335 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:30.422335 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:30.422335 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:30.422335 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:30.422335 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:30.422335 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:30.422335 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:30.422335 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:30.422335 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:30.422335 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:30.422335 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:30.422335 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:30.422335 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:30.422335 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:30.422335 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:03:30.423386 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:03:30.423386 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:30.423386 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:30.423386 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 332/644] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:03:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:30.423386 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13505aa1-4a9e-4a78-babf-92a4b71f3d14 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:30.445326 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8596e54d-8e04-487f-a911-b660a5149a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:30.446150 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8596e54d-8e04-487f-a911-b660a5149a24 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:30.446572 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8596e54d-8e04-487f-a911-b660a5149a24 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:30.447370 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8596e54d-8e04-487f-a911-b660a5149a24 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:30.448200 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8596e54d-8e04-487f-a911-b660a5149a24 None admin] RBAC: Adding query filter params (name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:30.448503 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8596e54d-8e04-487f-a911-b660a5149a24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:30.449937 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8596e54d-8e04-487f-a911-b660a5149a24 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:30.458778 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 313/645] 10.222.0.43 () {56 vars in 1103 bytes} [Tue Sep 6 16:03:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:30.461641 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8596e54d-8e04-487f-a911-b660a5149a24 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:30.493453 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:30.532565 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13505aa1-4a9e-4a78-babf-92a4b71f3d14 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:30.567648 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8596e54d-8e04-487f-a911-b660a5149a24 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:30.570960 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0393b2-1a9b-4dcc-8aed-a1c1f5c76e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:30.571583 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0393b2-1a9b-4dcc-8aed-a1c1f5c76e55 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:30.571857 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0393b2-1a9b-4dcc-8aed-a1c1f5c76e55 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:30.572130 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0393b2-1a9b-4dcc-8aed-a1c1f5c76e55 None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:30.572856 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0393b2-1a9b-4dcc-8aed-a1c1f5c76e55 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:30.573164 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0393b2-1a9b-4dcc-8aed-a1c1f5c76e55 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:30.574607 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0393b2-1a9b-4dcc-8aed-a1c1f5c76e55 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:30.590287 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 333/646] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 6 16:03:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:30.591093 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0393b2-1a9b-4dcc-8aed-a1c1f5c76e55 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:30.605205 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60860d3-f463-4845-9a9a-c7701f647188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:30.606060 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60860d3-f463-4845-9a9a-c7701f647188 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:30.606170 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60860d3-f463-4845-9a9a-c7701f647188 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:30.606388 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60860d3-f463-4845-9a9a-c7701f647188 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:30.617594 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60860d3-f463-4845-9a9a-c7701f647188 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:30.618012 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:30.619153 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60860d3-f463-4845-9a9a-c7701f647188 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:30.637116 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-b60860d3-f463-4845-9a9a-c7701f647188 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:30.637116 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:30.637116 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:30.637116 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:30.637116 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:30.637116 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:30.637116 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:30.637116 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:30.637116 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:30.637116 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:30.637116 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:30.637116 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:30.637116 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:03:30.637116 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:03:30.637116 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:03:30.637116 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:03:30.638162 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:30.638162 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:30.638162 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:30.638162 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:30.638162 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:30.638162 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:30.638162 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:30.638162 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:30.638162 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:30.638162 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:30.638162 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:30.638162 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:30.638162 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:30.638162 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:30.638162 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:30.638162 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:30.638162 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:03:30.644445 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:03:30.644445 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:30.644445 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:30.644445 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 314/647] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:03:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:30.644445 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60860d3-f463-4845-9a9a-c7701f647188 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:30.702738 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0393b2-1a9b-4dcc-8aed-a1c1f5c76e55 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:30.728853 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:30.740089 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316afa68-578d-4ec8-8fcc-74655e724f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:30.740699 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316afa68-578d-4ec8-8fcc-74655e724f37 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:30.741000 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316afa68-578d-4ec8-8fcc-74655e724f37 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:30.741275 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316afa68-578d-4ec8-8fcc-74655e724f37 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:30.741915 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316afa68-578d-4ec8-8fcc-74655e724f37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:30.742221 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316afa68-578d-4ec8-8fcc-74655e724f37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:30.743597 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316afa68-578d-4ec8-8fcc-74655e724f37 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:30.748706 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60860d3-f463-4845-9a9a-c7701f647188 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:30.755439 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 334/648] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:03:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:30.756144 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316afa68-578d-4ec8-8fcc-74655e724f37 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:30.787092 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2701b7a4-a342-4c94-ad74-f20632ebb61a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:30.787727 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2701b7a4-a342-4c94-ad74-f20632ebb61a None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:30.787995 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2701b7a4-a342-4c94-ad74-f20632ebb61a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:30.788260 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2701b7a4-a342-4c94-ad74-f20632ebb61a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:30.799310 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2701b7a4-a342-4c94-ad74-f20632ebb61a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:30.800827 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2701b7a4-a342-4c94-ad74-f20632ebb61a None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:30.815150 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-2701b7a4-a342-4c94-ad74-f20632ebb61a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:30.815150 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:30.815150 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:30.815150 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:30.815150 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:30.815150 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:30.815150 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:30.815150 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:30.815150 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:30.815150 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:30.815150 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:30.815150 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:30.815150 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:03:30.815150 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:03:30.815150 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:03:30.815150 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:03:30.816177 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:30.816177 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:30.816177 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:30.816177 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:30.816177 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:30.816177 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:30.816177 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:30.816177 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:30.816177 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:30.816177 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:30.816177 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:30.816177 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:30.816177 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:30.816177 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:30.816177 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:30.816177 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:30.816177 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:03:30.817145 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:03:30.817145 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:30.817145 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:30.817145 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 315/649] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:03:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:30.817377 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2701b7a4-a342-4c94-ad74-f20632ebb61a None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:30.844559 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:30.864644 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316afa68-578d-4ec8-8fcc-74655e724f37 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:30.899777 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d70c8f4-f718-41d2-9ead-a04909b4a276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:30.900400 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d70c8f4-f718-41d2-9ead-a04909b4a276 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:30.900670 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d70c8f4-f718-41d2-9ead-a04909b4a276 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:30.900986 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d70c8f4-f718-41d2-9ead-a04909b4a276 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:30.901720 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d70c8f4-f718-41d2-9ead-a04909b4a276 None admin] RBAC: Adding query filter params (role.id=31321627b02c4ff188f8fe1521ce3181, scope.project.id=128d210e8dad48ac921ed4cb1d444ceb, user.id=2c163b39e61648c385f065ce8091c3ab) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:30.902439 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d70c8f4-f718-41d2-9ead-a04909b4a276 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:30.904392 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d70c8f4-f718-41d2-9ead-a04909b4a276 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:30.910939 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 335/650] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 6 16:03:30 2022] GET /identity/v3/role_assignments?user.id=2c163b39e61648c385f065ce8091c3ab&scope.project.id=128d210e8dad48ac921ed4cb1d444ceb&role.id=31321627b02c4ff188f8fe1521ce3181 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:30.911729 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d70c8f4-f718-41d2-9ead-a04909b4a276 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:30.946363 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2701b7a4-a342-4c94-ad74-f20632ebb61a None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:30.992200 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:31.000330 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238ba659-c063-4f14-a201-cdc2dd8e566f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:31.002288 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238ba659-c063-4f14-a201-cdc2dd8e566f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:31.002780 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238ba659-c063-4f14-a201-cdc2dd8e566f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:31.003248 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238ba659-c063-4f14-a201-cdc2dd8e566f None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:31.004167 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238ba659-c063-4f14-a201-cdc2dd8e566f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:31.004648 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238ba659-c063-4f14-a201-cdc2dd8e566f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:31.006281 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238ba659-c063-4f14-a201-cdc2dd8e566f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:31.026648 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 316/651] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:03:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:31.028120 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238ba659-c063-4f14-a201-cdc2dd8e566f None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:31.042775 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d70c8f4-f718-41d2-9ead-a04909b4a276 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:31.087267 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7887ed58-20b7-435d-8179-a2a3350f162c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:31.088192 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7887ed58-20b7-435d-8179-a2a3350f162c None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:31.088468 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7887ed58-20b7-435d-8179-a2a3350f162c None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:31.088734 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7887ed58-20b7-435d-8179-a2a3350f162c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:31.099447 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7887ed58-20b7-435d-8179-a2a3350f162c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:31.100980 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7887ed58-20b7-435d-8179-a2a3350f162c None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:31.116201 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-7887ed58-20b7-435d-8179-a2a3350f162c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 16:03:31.116201 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:31.116201 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:31.116201 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:31.116201 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:31.116201 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:31.116201 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:31.116201 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:31.116201 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:31.116201 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:31.116201 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:31.116201 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:31.116201 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:03:31.116201 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:03:31.116201 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:03:31.116201 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:03:31.117229 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:31.117229 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:31.117229 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:03:31.117229 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:03:31.117229 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:03:31.117229 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:03:31.117229 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:31.117229 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:31.117229 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:31.117229 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:31.117229 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:31.117229 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:31.117229 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:31.117229 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:31.117229 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:31.117229 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:31.117229 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:31.117229 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:31.118779 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:31.118779 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:31.118779 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:03:31.118779 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:03:31.118779 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:03:31.118779 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:03:31.118779 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:03:31.118779 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:03:31.118779 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 16:03:31.118779 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:31.118779 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 336/652] 10.222.0.43 () {56 vars in 1093 bytes} [Tue Sep 6 16:03:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:31.119369 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7887ed58-20b7-435d-8179-a2a3350f162c None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:31.123828 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:31.179206 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238ba659-c063-4f14-a201-cdc2dd8e566f None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:31.221365 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9087d6d-061d-4ddd-bfe5-213c06ff70b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:31.221984 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9087d6d-061d-4ddd-bfe5-213c06ff70b6 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:31.222256 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9087d6d-061d-4ddd-bfe5-213c06ff70b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:31.222599 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9087d6d-061d-4ddd-bfe5-213c06ff70b6 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:31.223257 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9087d6d-061d-4ddd-bfe5-213c06ff70b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:31.223558 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9087d6d-061d-4ddd-bfe5-213c06ff70b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:31.225714 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9087d6d-061d-4ddd-bfe5-213c06ff70b6 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:31.237700 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 317/653] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:03:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:31.239290 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9087d6d-061d-4ddd-bfe5-213c06ff70b6 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:31.253101 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7887ed58-20b7-435d-8179-a2a3350f162c None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:31.291640 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bd013c-cf3b-4a69-a07d-e473f44f1460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:31.292328 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bd013c-cf3b-4a69-a07d-e473f44f1460 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:31.292602 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bd013c-cf3b-4a69-a07d-e473f44f1460 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:31.292914 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bd013c-cf3b-4a69-a07d-e473f44f1460 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:31.300623 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bd013c-cf3b-4a69-a07d-e473f44f1460 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:31.302159 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bd013c-cf3b-4a69-a07d-e473f44f1460 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:31.311597 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:31.314244 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-f0bd013c-cf3b-4a69-a07d-e473f44f1460 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 06 16:03:31.314244 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:31.314244 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:31.314244 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:31.314244 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:31.314244 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:31.314244 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:31.314244 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:31.314244 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:31.314244 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:31.314244 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:31.314244 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:31.314244 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:03:31.314244 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:03:31.314244 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:03:31.314244 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:03:31.315161 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:31.315161 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:31.315161 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:03:31.315161 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:03:31.315161 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:03:31.315161 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:03:31.315161 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:31.315161 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:31.315161 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:31.315161 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:31.315161 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:31.315161 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:31.315161 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:31.315161 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:31.315161 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:31.315161 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:31.315161 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:31.315161 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:31.316603 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:31.316603 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:31.316603 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:03:31.316603 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:03:31.316603 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:03:31.316603 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:03:31.316603 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:03:31.316603 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:03:31.316603 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 06 16:03:31.316603 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:31.316603 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 337/654] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 6 16:03:31 2022] GET /identity/v3/users/cinder => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:31.316603 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bd013c-cf3b-4a69-a07d-e473f44f1460 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:31.369641 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9087d6d-061d-4ddd-bfe5-213c06ff70b6 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:31.388560 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:31.409831 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca164f96-3aca-4237-ba29-7bea7ecaa8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:31.410298 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca164f96-3aca-4237-ba29-7bea7ecaa8f6 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:31.410575 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca164f96-3aca-4237-ba29-7bea7ecaa8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:31.410843 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca164f96-3aca-4237-ba29-7bea7ecaa8f6 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:31.411625 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca164f96-3aca-4237-ba29-7bea7ecaa8f6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:31.411879 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca164f96-3aca-4237-ba29-7bea7ecaa8f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:31.413471 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca164f96-3aca-4237-ba29-7bea7ecaa8f6 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:31.433102 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 318/655] 10.222.0.43 () {56 vars in 1133 bytes} [Tue Sep 6 16:03:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:31.433968 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca164f96-3aca-4237-ba29-7bea7ecaa8f6 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:31.434512 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bd013c-cf3b-4a69-a07d-e473f44f1460 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:31.471062 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02909181-0f7e-4919-9b0a-e4db2d86fdb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:31.472086 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02909181-0f7e-4919-9b0a-e4db2d86fdb3 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:31.472623 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02909181-0f7e-4919-9b0a-e4db2d86fdb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:31.473156 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02909181-0f7e-4919-9b0a-e4db2d86fdb3 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:31.481037 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02909181-0f7e-4919-9b0a-e4db2d86fdb3 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:31.482600 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02909181-0f7e-4919-9b0a-e4db2d86fdb3 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:31.493829 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-02909181-0f7e-4919-9b0a-e4db2d86fdb3 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 06 16:03:31.493829 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:31.493829 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:31.493829 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:31.493829 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:31.493829 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:31.493829 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:31.493829 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:31.493829 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:31.493829 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:31.493829 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:31.493829 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:31.493829 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:03:31.493829 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:03:31.493829 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:03:31.493829 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:03:31.494799 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:31.494799 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:31.494799 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:03:31.494799 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:03:31.494799 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:03:31.494799 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:03:31.494799 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:31.494799 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:31.494799 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:31.494799 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:31.494799 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:31.494799 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:31.494799 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:31.494799 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:31.494799 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:31.494799 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:31.494799 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:31.494799 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:31.495731 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:31.495731 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:31.495731 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:03:31.495731 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:03:31.495731 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:03:31.495731 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:03:31.495731 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:03:31.495731 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:03:31.495731 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 06 16:03:31.495731 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:31.495731 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 338/656] 10.222.0.43 () {56 vars in 1102 bytes} [Tue Sep 6 16:03:31 2022] GET /identity/v3/users/neutron => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:31.495731 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02909181-0f7e-4919-9b0a-e4db2d86fdb3 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:31.542626 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca164f96-3aca-4237-ba29-7bea7ecaa8f6 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:31.554055 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:31.579955 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a6300e-7b16-42f7-8097-564869bd2e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:31.580584 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a6300e-7b16-42f7-8097-564869bd2e43 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:31.580886 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a6300e-7b16-42f7-8097-564869bd2e43 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:31.581155 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a6300e-7b16-42f7-8097-564869bd2e43 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:31.581964 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a6300e-7b16-42f7-8097-564869bd2e43 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:31.582263 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a6300e-7b16-42f7-8097-564869bd2e43 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:31.583692 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a6300e-7b16-42f7-8097-564869bd2e43 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:31.605832 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 319/657] 10.222.0.43 () {56 vars in 1137 bytes} [Tue Sep 6 16:03:31 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:31.605832 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c8749a-96ce-4fef-b061-4e75b3411051 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:31.605832 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c8749a-96ce-4fef-b061-4e75b3411051 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:31.606179 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02909181-0f7e-4919-9b0a-e4db2d86fdb3 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:31.606260 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c8749a-96ce-4fef-b061-4e75b3411051 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:31.608350 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 320/658] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:03:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:03:31.609273 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c8749a-96ce-4fef-b061-4e75b3411051 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:31.631319 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:31.643931 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8939db-bfe9-4cce-b9dd-d06ccea3baba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:31.644589 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8939db-bfe9-4cce-b9dd-d06ccea3baba None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:31.645506 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8939db-bfe9-4cce-b9dd-d06ccea3baba None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:31.645506 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8939db-bfe9-4cce-b9dd-d06ccea3baba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:31.712918 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c8749a-96ce-4fef-b061-4e75b3411051 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:31.714039 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8939db-bfe9-4cce-b9dd-d06ccea3baba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:31.715501 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8939db-bfe9-4cce-b9dd-d06ccea3baba None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:31.746641 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e439114-01e7-407e-8518-2e0a136581b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:31.747253 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e439114-01e7-407e-8518-2e0a136581b1 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:31.747528 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e439114-01e7-407e-8518-2e0a136581b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:31.747804 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e439114-01e7-407e-8518-2e0a136581b1 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:31.748538 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e439114-01e7-407e-8518-2e0a136581b1 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:31.748859 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e439114-01e7-407e-8518-2e0a136581b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:31.750318 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e439114-01e7-407e-8518-2e0a136581b1 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:31.771826 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 321/659] 10.222.0.43 () {56 vars in 1139 bytes} [Tue Sep 6 16:03:31 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:31.771826 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e439114-01e7-407e-8518-2e0a136581b1 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:31.796585 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:31.824216 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 339/660] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 6 16:03:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 16:03:31.826032 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e64ad4-79a6-49df-87c3-caae1c14630e None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:31.826579 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e64ad4-79a6-49df-87c3-caae1c14630e None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:31.827338 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e64ad4-79a6-49df-87c3-caae1c14630e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:31.848035 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-a2e64ad4-79a6-49df-87c3-caae1c14630e None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:03:31.849613 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a2e64ad4-79a6-49df-87c3-caae1c14630e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:03:31.867938 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:31.872864 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e439114-01e7-407e-8518-2e0a136581b1 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:31.908880 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acf23fc-7f3d-4c5b-90d9-2ec585056325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:31.909462 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acf23fc-7f3d-4c5b-90d9-2ec585056325 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:31.909743 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acf23fc-7f3d-4c5b-90d9-2ec585056325 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:31.910037 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acf23fc-7f3d-4c5b-90d9-2ec585056325 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:31.924791 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 340/661] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:03:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:03:31.924791 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e64ad4-79a6-49df-87c3-caae1c14630e None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:31.985788 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acf23fc-7f3d-4c5b-90d9-2ec585056325 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:31.987288 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acf23fc-7f3d-4c5b-90d9-2ec585056325 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:32.016181 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:32.051784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e64ad4-79a6-49df-87c3-caae1c14630e None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:32.093646 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235d0d8e-640f-4909-88aa-a5397b08d603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:32.094261 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235d0d8e-640f-4909-88aa-a5397b08d603 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:32.094537 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235d0d8e-640f-4909-88aa-a5397b08d603 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:32.094808 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235d0d8e-640f-4909-88aa-a5397b08d603 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:32.100120 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 322/662] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 6 16:03:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 16:03:32.101047 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acf23fc-7f3d-4c5b-90d9-2ec585056325 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:32.125113 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:32.179916 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235d0d8e-640f-4909-88aa-a5397b08d603 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:32.181356 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235d0d8e-640f-4909-88aa-a5397b08d603 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:32.217296 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acf23fc-7f3d-4c5b-90d9-2ec585056325 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:32.245130 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:32.258159 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9e9c3b-3db2-409c-82ea-2014434a3e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:32.259194 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e9c3b-3db2-409c-82ea-2014434a3e4b None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:32.259472 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e9c3b-3db2-409c-82ea-2014434a3e4b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:32.259744 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e9c3b-3db2-409c-82ea-2014434a3e4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:32.271827 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9e9c3b-3db2-409c-82ea-2014434a3e4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:32.274973 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9e9c3b-3db2-409c-82ea-2014434a3e4b None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:32.283931 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 341/663] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 6 16:03:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 16:03:32.285333 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235d0d8e-640f-4909-88aa-a5397b08d603 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:32.294901 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-3b9e9c3b-3db2-409c-82ea-2014434a3e4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:32.294901 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:32.294901 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:32.294901 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:32.294901 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:32.294901 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:32.294901 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:32.294901 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:32.294901 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:32.294901 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:32.294901 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:32.294901 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:32.294901 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:03:32.294901 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:03:32.294901 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:03:32.294901 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:03:32.295937 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:32.295937 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:32.295937 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:32.295937 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:32.295937 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:32.295937 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:32.295937 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:32.295937 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:32.295937 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:32.295937 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:32.295937 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:32.295937 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:32.295937 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:32.295937 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:32.295937 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:32.295937 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:32.295937 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:03:32.296882 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:03:32.296882 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:32.296882 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:32.300965 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 323/664] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:03:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:32.301798 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9e9c3b-3db2-409c-82ea-2014434a3e4b None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:32.344245 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:32.390096 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235d0d8e-640f-4909-88aa-a5397b08d603 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:32.409795 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9e9c3b-3db2-409c-82ea-2014434a3e4b None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:32.427757 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e36975-6ced-4b8b-ae02-7303128a1517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:32.428398 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e36975-6ced-4b8b-ae02-7303128a1517 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:32.428674 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e36975-6ced-4b8b-ae02-7303128a1517 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:32.428973 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e36975-6ced-4b8b-ae02-7303128a1517 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:32.439131 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e36975-6ced-4b8b-ae02-7303128a1517 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:32.440516 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e36975-6ced-4b8b-ae02-7303128a1517 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:32.446017 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be073a5-896b-4c2a-82ab-ae2880e9870e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:32.446863 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be073a5-896b-4c2a-82ab-ae2880e9870e None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:32.447363 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be073a5-896b-4c2a-82ab-ae2880e9870e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:32.447815 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be073a5-896b-4c2a-82ab-ae2880e9870e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:32.455374 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-18e36975-6ced-4b8b-ae02-7303128a1517 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:32.455374 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:32.455374 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:32.455374 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:32.455374 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:32.455374 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:32.455374 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:32.455374 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:32.455374 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:32.455374 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:32.455374 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:32.455374 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:32.455374 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:03:32.455374 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:03:32.455374 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:03:32.455374 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:03:32.456285 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:32.456285 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:32.456285 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:32.456285 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:32.456285 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:32.456285 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:32.456285 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:32.456285 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:32.456285 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:32.456285 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:32.456285 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:32.456285 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:32.456285 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:32.456285 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:32.456285 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:32.456285 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:32.456285 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:03:32.457265 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:03:32.457265 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:32.457265 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:32.457265 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 342/665] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:03:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:32.457495 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e36975-6ced-4b8b-ae02-7303128a1517 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:32.464599 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be073a5-896b-4c2a-82ab-ae2880e9870e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:03:32.469999 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-2be073a5-896b-4c2a-82ab-ae2880e9870e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:03:32.469999 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:32.469999 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:32.469999 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:32.469999 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:32.469999 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:32.469999 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:32.469999 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:32.469999 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:32.469999 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:32.469999 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:32.469999 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:32.469999 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:03:32.469999 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:03:32.469999 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:03:32.469999 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:03:32.470948 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:03:32.470948 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:03:32.470948 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:03:32.470948 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:03:32.470948 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:32.470948 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:32.470948 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:32.470948 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:32.470948 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:32.470948 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:32.470948 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:32.470948 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:32.470948 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:32.470948 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:32.470948 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:32.470948 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:32.475578 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:32.475578 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:32.475578 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:32.475578 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:32.475578 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:03:32.475578 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:03:32.475578 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:03:32.475578 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:03:32.475578 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:03:32.475578 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:03:32.475578 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:03:32.475578 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:32.475578 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 324/666] 10.222.0.43 () {56 vars in 1102 bytes} [Tue Sep 6 16:03:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:32.475578 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be073a5-896b-4c2a-82ab-ae2880e9870e None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:32.518167 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:32.568787 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e36975-6ced-4b8b-ae02-7303128a1517 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:32.576964 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:32.581339 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be073a5-896b-4c2a-82ab-ae2880e9870e None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:32.607076 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9fa9d9-5bdb-44c2-af7e-f82f31b99c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:32.607866 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9fa9d9-5bdb-44c2-af7e-f82f31b99c1b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:32.608139 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9fa9d9-5bdb-44c2-af7e-f82f31b99c1b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:32.608720 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9fa9d9-5bdb-44c2-af7e-f82f31b99c1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:32.616689 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38387dd8-257c-4f81-bd71-b6735b35036d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:32.617333 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38387dd8-257c-4f81-bd71-b6735b35036d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:32.617612 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38387dd8-257c-4f81-bd71-b6735b35036d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:32.618264 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38387dd8-257c-4f81-bd71-b6735b35036d None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:32.618912 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38387dd8-257c-4f81-bd71-b6735b35036d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:32.619215 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38387dd8-257c-4f81-bd71-b6735b35036d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:32.619798 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9fa9d9-5bdb-44c2-af7e-f82f31b99c1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:32.620642 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38387dd8-257c-4f81-bd71-b6735b35036d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:32.621260 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9fa9d9-5bdb-44c2-af7e-f82f31b99c1b None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:32.632996 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 325/667] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:03:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:32.634208 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38387dd8-257c-4f81-bd71-b6735b35036d None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:32.641957 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-4c9fa9d9-5bdb-44c2-af7e-f82f31b99c1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:32.641957 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:32.641957 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:32.641957 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:32.641957 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:32.641957 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:32.641957 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:32.641957 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:32.641957 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:32.641957 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:32.641957 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:32.641957 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:32.641957 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:03:32.641957 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:03:32.641957 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:03:32.641957 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:03:32.643126 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:32.643126 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:32.643126 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:32.643126 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:32.643126 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:32.643126 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:32.643126 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:32.643126 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:32.643126 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:32.643126 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:32.643126 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:32.643126 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:32.643126 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:32.643126 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:32.643126 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:32.643126 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:32.643126 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:03:32.644030 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:03:32.644030 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:32.644030 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:32.644030 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 343/668] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:03:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:32.644030 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9fa9d9-5bdb-44c2-af7e-f82f31b99c1b None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:32.742271 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:32.746373 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38387dd8-257c-4f81-bd71-b6735b35036d None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:32.750755 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9fa9d9-5bdb-44c2-af7e-f82f31b99c1b None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:32.783876 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9d1b19-3542-4e3c-b272-a9ff800b6e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:32.784350 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84dd757-8f83-4cd3-aa9e-12017da2da2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:32.784534 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9d1b19-3542-4e3c-b272-a9ff800b6e6e None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:32.784875 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9d1b19-3542-4e3c-b272-a9ff800b6e6e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:32.785078 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84dd757-8f83-4cd3-aa9e-12017da2da2d None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:32.785184 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9d1b19-3542-4e3c-b272-a9ff800b6e6e None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:32.785332 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84dd757-8f83-4cd3-aa9e-12017da2da2d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:32.785574 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84dd757-8f83-4cd3-aa9e-12017da2da2d None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:32.785833 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9d1b19-3542-4e3c-b272-a9ff800b6e6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:32.786144 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9d1b19-3542-4e3c-b272-a9ff800b6e6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:32.786233 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84dd757-8f83-4cd3-aa9e-12017da2da2d None admin] RBAC: Adding query filter params (name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:32.786524 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84dd757-8f83-4cd3-aa9e-12017da2da2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:32.787537 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9d1b19-3542-4e3c-b272-a9ff800b6e6e None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:32.787886 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84dd757-8f83-4cd3-aa9e-12017da2da2d None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:32.801411 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 344/669] 10.222.0.43 () {56 vars in 1103 bytes} [Tue Sep 6 16:03:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:32.801411 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84dd757-8f83-4cd3-aa9e-12017da2da2d None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:32.801666 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 326/670] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:03:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:32.801666 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9d1b19-3542-4e3c-b272-a9ff800b6e6e None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:32.825847 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:32.907258 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9d1b19-3542-4e3c-b272-a9ff800b6e6e None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:32.913001 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84dd757-8f83-4cd3-aa9e-12017da2da2d None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:32.941335 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8851a2c3-9051-4bb9-98da-02d8f03f2998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:32.942187 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851a2c3-9051-4bb9-98da-02d8f03f2998 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:32.942327 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851a2c3-9051-4bb9-98da-02d8f03f2998 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:32.942505 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851a2c3-9051-4bb9-98da-02d8f03f2998 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:32.943129 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8851a2c3-9051-4bb9-98da-02d8f03f2998 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:32.943488 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8851a2c3-9051-4bb9-98da-02d8f03f2998 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:32.944927 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8851a2c3-9051-4bb9-98da-02d8f03f2998 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:32.951438 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ee30c1-3c7e-4044-8b3d-179c5ce2a835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:32.952080 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ee30c1-3c7e-4044-8b3d-179c5ce2a835 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:32.952358 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ee30c1-3c7e-4044-8b3d-179c5ce2a835 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:32.953164 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ee30c1-3c7e-4044-8b3d-179c5ce2a835 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:32.958426 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 327/671] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:03:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:32.961292 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ee30c1-3c7e-4044-8b3d-179c5ce2a835 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:32.961886 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8851a2c3-9051-4bb9-98da-02d8f03f2998 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:32.962749 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ee30c1-3c7e-4044-8b3d-179c5ce2a835 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:32.974499 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-d7ee30c1-3c7e-4044-8b3d-179c5ce2a835 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:03:32.974499 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:32.974499 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:32.974499 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:32.974499 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:32.974499 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:32.974499 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:32.974499 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:32.974499 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:32.974499 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:32.974499 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:32.974499 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:32.974499 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:03:32.974499 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:03:32.974499 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:03:32.975511 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:03:32.975511 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:32.975511 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:32.975511 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:32.975511 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:32.975511 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:32.975511 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:32.975511 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:32.975511 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:32.975511 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:32.975511 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:32.975511 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:32.975511 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:32.975511 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:32.975511 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:32.975511 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:32.975511 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:32.976425 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:03:32.976425 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:03:32.976425 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:03:32.976425 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:03:32.976425 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:03:32.976425 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:03:32.976425 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:03:32.976425 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:32.980556 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 345/672] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 6 16:03:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:32.981256 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ee30c1-3c7e-4044-8b3d-179c5ce2a835 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:32.984188 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:33.060651 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:33.063810 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8851a2c3-9051-4bb9-98da-02d8f03f2998 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:33.079896 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ee30c1-3c7e-4044-8b3d-179c5ce2a835 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:33.097937 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a09c0bb-d365-418d-92f4-7e5d7f85557b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:33.098564 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a09c0bb-d365-418d-92f4-7e5d7f85557b None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:33.098843 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a09c0bb-d365-418d-92f4-7e5d7f85557b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:33.099113 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a09c0bb-d365-418d-92f4-7e5d7f85557b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:33.108973 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a09c0bb-d365-418d-92f4-7e5d7f85557b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:33.110452 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a09c0bb-d365-418d-92f4-7e5d7f85557b None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:33.113565 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7287cd32-0413-46a7-a958-70a6c388ed4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:33.114186 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7287cd32-0413-46a7-a958-70a6c388ed4b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:33.114464 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7287cd32-0413-46a7-a958-70a6c388ed4b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:33.114821 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7287cd32-0413-46a7-a958-70a6c388ed4b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:33.121493 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7287cd32-0413-46a7-a958-70a6c388ed4b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:33.122930 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7287cd32-0413-46a7-a958-70a6c388ed4b None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:33.125850 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-0a09c0bb-d365-418d-92f4-7e5d7f85557b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:33.125850 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:33.125850 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:33.125850 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:33.125850 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:33.125850 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:33.125850 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:33.125850 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:33.125850 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:33.125850 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:33.125850 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:33.125850 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:33.125850 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:03:33.125850 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:03:33.125850 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:03:33.125850 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:03:33.126931 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:33.126931 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:33.126931 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:33.126931 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:33.126931 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:33.126931 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:33.126931 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:33.126931 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:33.126931 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:33.126931 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:33.126931 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:33.126931 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:33.126931 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:33.126931 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:33.126931 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:33.126931 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:33.126931 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:03:33.127812 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:03:33.127812 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:33.127812 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:33.127812 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 328/673] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:03:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:33.127812 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a09c0bb-d365-418d-92f4-7e5d7f85557b None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:33.134077 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-7287cd32-0413-46a7-a958-70a6c388ed4b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:03:33.134077 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:33.134077 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:33.134077 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:33.134077 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:33.134077 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:33.134077 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:33.134077 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:33.134077 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:33.134077 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:33.134077 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:33.134077 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:33.134077 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:03:33.134077 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:03:33.134077 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:03:33.135093 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:03:33.135093 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:33.135093 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:33.135093 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:33.135093 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:33.135093 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:33.135093 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:33.135093 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:33.135093 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:33.135093 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:33.135093 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:33.135093 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:33.135093 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:33.135093 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:33.135093 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:33.135093 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:33.135093 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:33.136044 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:03:33.136044 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:03:33.136044 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:03:33.136044 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:03:33.136044 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:03:33.136044 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:03:33.136044 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:03:33.136044 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:33.139802 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 346/674] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 6 16:03:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:33.140507 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7287cd32-0413-46a7-a958-70a6c388ed4b None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:33.196979 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:33.228887 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a09c0bb-d365-418d-92f4-7e5d7f85557b None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:33.241080 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7287cd32-0413-46a7-a958-70a6c388ed4b None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:33.266464 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4254613-d35b-47ce-92cd-a3a3db4d76ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:33.267427 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4254613-d35b-47ce-92cd-a3a3db4d76ff None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:33.267427 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4254613-d35b-47ce-92cd-a3a3db4d76ff None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:33.267621 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4254613-d35b-47ce-92cd-a3a3db4d76ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:33.275329 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4254613-d35b-47ce-92cd-a3a3db4d76ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:33.276820 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4254613-d35b-47ce-92cd-a3a3db4d76ff None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:33.279062 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befcc11e-5124-469e-8902-57529a469337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:33.279673 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befcc11e-5124-469e-8902-57529a469337 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:33.279944 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befcc11e-5124-469e-8902-57529a469337 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:33.280211 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befcc11e-5124-469e-8902-57529a469337 None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:33.280932 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befcc11e-5124-469e-8902-57529a469337 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:33.281231 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befcc11e-5124-469e-8902-57529a469337 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:33.282662 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befcc11e-5124-469e-8902-57529a469337 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:33.288286 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-c4254613-d35b-47ce-92cd-a3a3db4d76ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:03:33.288286 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:33.288286 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:33.288286 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:33.288286 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:33.288286 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:33.288286 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:33.288286 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:33.288286 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:33.288286 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:33.288286 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:33.288286 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:33.288286 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:03:33.288286 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:03:33.288286 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:03:33.289257 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:03:33.289257 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:33.289257 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:33.289257 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:33.289257 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:33.289257 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:33.289257 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:33.289257 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:33.289257 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:33.289257 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:33.289257 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:33.289257 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:33.289257 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:33.289257 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:33.289257 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:33.289257 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:33.289257 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:33.290153 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:03:33.290153 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:03:33.290153 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:03:33.290153 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:03:33.290153 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:03:33.290153 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:03:33.290153 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:03:33.290153 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:33.290153 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 329/675] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 6 16:03:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:33.290663 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609670f1-c320-44ad-af2d-677e37fc51aa None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:33.291497 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609670f1-c320-44ad-af2d-677e37fc51aa None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:33.291497 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609670f1-c320-44ad-af2d-677e37fc51aa None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:33.296684 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 330/676] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:03:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:03:33.296684 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609670f1-c320-44ad-af2d-677e37fc51aa None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:33.297433 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:33.306407 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 347/677] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 6 16:03:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:33.306407 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befcc11e-5124-469e-8902-57529a469337 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:33.395609 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609670f1-c320-44ad-af2d-677e37fc51aa None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:33.399618 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befcc11e-5124-469e-8902-57529a469337 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:33.424317 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:33.431677 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166d52a3-aa26-466a-9cb7-1fc5db380ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:33.432284 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166d52a3-aa26-466a-9cb7-1fc5db380ff5 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:33.432562 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166d52a3-aa26-466a-9cb7-1fc5db380ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:33.432679 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cf023d-9d1f-4a7a-8a15-f693155c582c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:33.432866 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166d52a3-aa26-466a-9cb7-1fc5db380ff5 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:33.433291 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf023d-9d1f-4a7a-8a15-f693155c582c None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:33.433514 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166d52a3-aa26-466a-9cb7-1fc5db380ff5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:33.433601 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf023d-9d1f-4a7a-8a15-f693155c582c None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:33.433813 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166d52a3-aa26-466a-9cb7-1fc5db380ff5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:33.433901 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf023d-9d1f-4a7a-8a15-f693155c582c None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:33.434494 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cf023d-9d1f-4a7a-8a15-f693155c582c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:33.434792 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cf023d-9d1f-4a7a-8a15-f693155c582c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:33.435186 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166d52a3-aa26-466a-9cb7-1fc5db380ff5 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:33.436187 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cf023d-9d1f-4a7a-8a15-f693155c582c None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:33.446940 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 331/678] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:03:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:33.451940 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 348/679] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 6 16:03:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:33.452077 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166d52a3-aa26-466a-9cb7-1fc5db380ff5 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:33.452171 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5ebdc0-cb81-451d-ab5e-4cd473162299 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:33.452171 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5ebdc0-cb81-451d-ab5e-4cd473162299 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:33.452171 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5ebdc0-cb81-451d-ab5e-4cd473162299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:33.467367 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-5f5ebdc0-cb81-451d-ab5e-4cd473162299 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:03:33.468543 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5f5ebdc0-cb81-451d-ab5e-4cd473162299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:03:33.521641 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:33.533212 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 349/680] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:03:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:03:33.534192 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5ebdc0-cb81-451d-ab5e-4cd473162299 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:33.550145 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166d52a3-aa26-466a-9cb7-1fc5db380ff5 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:33.584526 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24c2ce9-d67c-4359-b22a-ad50c993f58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:33.585167 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c2ce9-d67c-4359-b22a-ad50c993f58b None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:33.585437 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c2ce9-d67c-4359-b22a-ad50c993f58b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:33.585701 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c2ce9-d67c-4359-b22a-ad50c993f58b None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:33.586396 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24c2ce9-d67c-4359-b22a-ad50c993f58b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:33.586710 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24c2ce9-d67c-4359-b22a-ad50c993f58b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:33.588138 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24c2ce9-d67c-4359-b22a-ad50c993f58b None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:33.603948 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 332/681] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 6 16:03:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:33.603948 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24c2ce9-d67c-4359-b22a-ad50c993f58b None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:33.646332 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:33.650815 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5ebdc0-cb81-451d-ab5e-4cd473162299 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:33.686046 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668d9a61-d742-424e-909c-e38de03657ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:33.686659 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668d9a61-d742-424e-909c-e38de03657ef None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:33.687303 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668d9a61-d742-424e-909c-e38de03657ef None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:33.687303 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668d9a61-d742-424e-909c-e38de03657ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:33.687921 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668d9a61-d742-424e-909c-e38de03657ef None admin] RBAC: Adding query filter params (role.id=31321627b02c4ff188f8fe1521ce3181, scope.project.id=128d210e8dad48ac921ed4cb1d444ceb, user.id=63de41f4348243998029f7d07aa5dc40) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:33.688229 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668d9a61-d742-424e-909c-e38de03657ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:33.689712 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668d9a61-d742-424e-909c-e38de03657ef None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:33.696338 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 350/682] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 6 16:03:33 2022] GET /identity/v3/role_assignments?user.id=63de41f4348243998029f7d07aa5dc40&scope.project.id=128d210e8dad48ac921ed4cb1d444ceb&role.id=31321627b02c4ff188f8fe1521ce3181 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:33.697153 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668d9a61-d742-424e-909c-e38de03657ef None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:33.717143 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24c2ce9-d67c-4359-b22a-ad50c993f58b None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:33.759523 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799b99e5-174f-44c0-9915-3fa85b1b377e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:33.760602 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b99e5-174f-44c0-9915-3fa85b1b377e None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:33.761103 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b99e5-174f-44c0-9915-3fa85b1b377e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:33.761567 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b99e5-174f-44c0-9915-3fa85b1b377e None admin] PATH_INFO: `/v3/users/swift` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:33.764183 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:33.772202 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799b99e5-174f-44c0-9915-3fa85b1b377e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:33.774162 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799b99e5-174f-44c0-9915-3fa85b1b377e None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:33.786974 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-799b99e5-174f-44c0-9915-3fa85b1b377e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 06 16:03:33.786974 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:33.786974 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:33.786974 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:33.786974 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:33.786974 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:33.786974 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:33.786974 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:33.786974 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:33.786974 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:33.786974 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:33.786974 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:33.786974 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:03:33.786974 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:03:33.786974 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:03:33.786974 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:03:33.787990 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:33.787990 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:33.787990 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:03:33.787990 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:03:33.787990 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:03:33.787990 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:03:33.787990 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:33.787990 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:33.787990 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:33.787990 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:33.787990 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:33.787990 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:33.787990 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:33.787990 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:33.787990 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:33.787990 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:33.787990 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:33.787990 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:33.788961 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:33.788961 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:33.788961 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:03:33.788961 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:03:33.788961 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:03:33.788961 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:03:33.788961 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:03:33.788961 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:03:33.788961 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 06 16:03:33.788961 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:33.796187 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 333/683] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 6 16:03:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:33.797499 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799b99e5-174f-44c0-9915-3fa85b1b377e None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:33.817934 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668d9a61-d742-424e-909c-e38de03657ef None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:33.858909 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb075a2d-ac14-4793-82d4-23762532c8f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:33.859519 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb075a2d-ac14-4793-82d4-23762532c8f1 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:33.859806 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb075a2d-ac14-4793-82d4-23762532c8f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:33.860079 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb075a2d-ac14-4793-82d4-23762532c8f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:33.860838 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb075a2d-ac14-4793-82d4-23762532c8f1 None admin] RBAC: Adding query filter params (role.id=31321627b02c4ff188f8fe1521ce3181, scope.project.id=128d210e8dad48ac921ed4cb1d444ceb, user.id=d1b05748e4284153b0728164cd508dc9) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:33.861141 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb075a2d-ac14-4793-82d4-23762532c8f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:33.862571 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb075a2d-ac14-4793-82d4-23762532c8f1 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:33.868919 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 351/684] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 6 16:03:33 2022] GET /identity/v3/role_assignments?user.id=d1b05748e4284153b0728164cd508dc9&scope.project.id=128d210e8dad48ac921ed4cb1d444ceb&role.id=31321627b02c4ff188f8fe1521ce3181 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:33.869738 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb075a2d-ac14-4793-82d4-23762532c8f1 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:33.943853 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799b99e5-174f-44c0-9915-3fa85b1b377e None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:33.957783 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:33.989559 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b23a36-dd93-451b-92e3-b0834b0424b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:33.991007 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b23a36-dd93-451b-92e3-b0834b0424b6 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:33.991520 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b23a36-dd93-451b-92e3-b0834b0424b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:33.992222 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b23a36-dd93-451b-92e3-b0834b0424b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:33.993467 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b23a36-dd93-451b-92e3-b0834b0424b6 None admin] RBAC: Adding query filter params (role.id=31321627b02c4ff188f8fe1521ce3181, scope.project.id=128d210e8dad48ac921ed4cb1d444ceb, user.id=10565aefeaaa40549ed38bb9e0f4b189) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:33.993924 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b23a36-dd93-451b-92e3-b0834b0424b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:33.996233 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b23a36-dd93-451b-92e3-b0834b0424b6 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:34.004660 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 334/685] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 6 16:03:33 2022] GET /identity/v3/role_assignments?user.id=10565aefeaaa40549ed38bb9e0f4b189&scope.project.id=128d210e8dad48ac921ed4cb1d444ceb&role.id=31321627b02c4ff188f8fe1521ce3181 => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:34.005960 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b23a36-dd93-451b-92e3-b0834b0424b6 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:34.032451 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb075a2d-ac14-4793-82d4-23762532c8f1 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:34.055938 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:34.077646 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdc3f40-e00c-43dd-afdf-618c3d5e29e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:34.078269 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdc3f40-e00c-43dd-afdf-618c3d5e29e4 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:34.079201 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdc3f40-e00c-43dd-afdf-618c3d5e29e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:34.079487 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdc3f40-e00c-43dd-afdf-618c3d5e29e4 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:34.081037 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdc3f40-e00c-43dd-afdf-618c3d5e29e4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:34.081349 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdc3f40-e00c-43dd-afdf-618c3d5e29e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:34.083642 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdc3f40-e00c-43dd-afdf-618c3d5e29e4 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:34.108388 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 352/686] 10.222.0.43 () {56 vars in 1135 bytes} [Tue Sep 6 16:03:33 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:34.114141 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdc3f40-e00c-43dd-afdf-618c3d5e29e4 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:34.136998 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b23a36-dd93-451b-92e3-b0834b0424b6 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:34.180237 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d022e9e2-4b7b-41fd-bcbe-edc809b90329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:34.180950 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d022e9e2-4b7b-41fd-bcbe-edc809b90329 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:34.181227 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d022e9e2-4b7b-41fd-bcbe-edc809b90329 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:34.181498 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d022e9e2-4b7b-41fd-bcbe-edc809b90329 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:34.194538 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d022e9e2-4b7b-41fd-bcbe-edc809b90329 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:34.196171 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d022e9e2-4b7b-41fd-bcbe-edc809b90329 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:34.214142 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-d022e9e2-4b7b-41fd-bcbe-edc809b90329 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:34.214142 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:34.214142 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:34.214142 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:34.214142 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:34.214142 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:34.214142 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:34.214142 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:34.214142 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:34.214142 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:34.214142 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:34.214142 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:34.214142 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:03:34.214142 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:03:34.214142 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:03:34.214142 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:03:34.215136 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:34.215136 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:34.215136 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:34.215136 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:34.215136 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:34.215136 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:34.215136 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:34.215136 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:34.215136 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:34.215136 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:34.215136 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:34.215136 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:34.215136 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:34.215136 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:34.215136 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:34.215136 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:34.215136 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:03:34.216006 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:03:34.216006 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:34.216006 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:34.218134 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 335/687] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:03:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:34.225882 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d022e9e2-4b7b-41fd-bcbe-edc809b90329 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:34.254033 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdc3f40-e00c-43dd-afdf-618c3d5e29e4 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:34.270151 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:34.297875 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de3ee7e-0232-4e02-b975-4286ea4f1410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:34.298536 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de3ee7e-0232-4e02-b975-4286ea4f1410 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:34.298815 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de3ee7e-0232-4e02-b975-4286ea4f1410 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:34.299094 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de3ee7e-0232-4e02-b975-4286ea4f1410 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:34.351316 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:34.356227 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d022e9e2-4b7b-41fd-bcbe-edc809b90329 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:34.382523 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de3ee7e-0232-4e02-b975-4286ea4f1410 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:34.384279 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de3ee7e-0232-4e02-b975-4286ea4f1410 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:34.394731 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831fc471-b4f1-45f6-a90d-68094902f6af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:34.395353 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831fc471-b4f1-45f6-a90d-68094902f6af None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:34.395628 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831fc471-b4f1-45f6-a90d-68094902f6af None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:34.395897 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831fc471-b4f1-45f6-a90d-68094902f6af None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:34.396549 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831fc471-b4f1-45f6-a90d-68094902f6af None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:34.396927 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831fc471-b4f1-45f6-a90d-68094902f6af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:34.398390 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831fc471-b4f1-45f6-a90d-68094902f6af None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:34.412718 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 336/688] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:03:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:34.418596 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831fc471-b4f1-45f6-a90d-68094902f6af None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:34.516666 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:34.518572 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 353/689] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 6 16:03:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 06 16:03:34.524882 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de3ee7e-0232-4e02-b975-4286ea4f1410 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:34.538519 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831fc471-b4f1-45f6-a90d-68094902f6af None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:34.569001 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11511484-5567-4a5a-8cca-a0368013b091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:34.569625 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11511484-5567-4a5a-8cca-a0368013b091 None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:34.569892 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11511484-5567-4a5a-8cca-a0368013b091 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:34.570164 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11511484-5567-4a5a-8cca-a0368013b091 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:34.570817 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11511484-5567-4a5a-8cca-a0368013b091 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:34.573796 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11511484-5567-4a5a-8cca-a0368013b091 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:34.609132 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 337/690] 10.222.0.43 () {60 vars in 1132 bytes} [Tue Sep 6 16:03:34 2022] POST /identity/v3/users => generated 254 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:03:34.634657 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:34.676924 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de3ee7e-0232-4e02-b975-4286ea4f1410 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:34.733653 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1996571-a789-471b-a534-d07eaffb8667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:34.734322 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1996571-a789-471b-a534-d07eaffb8667 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:34.734600 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1996571-a789-471b-a534-d07eaffb8667 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:34.734875 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1996571-a789-471b-a534-d07eaffb8667 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:34.746760 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1996571-a789-471b-a534-d07eaffb8667 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:34.748239 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1996571-a789-471b-a534-d07eaffb8667 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:34.768651 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-b1996571-a789-471b-a534-d07eaffb8667 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:34.768651 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:34.768651 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:34.768651 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:34.768651 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:34.768651 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:34.768651 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:34.768651 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:34.768651 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:34.768651 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:34.768651 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:34.768651 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:34.768651 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:03:34.768651 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:03:34.768651 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:03:34.768651 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:03:34.769761 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:34.769761 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:34.769761 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:34.769761 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:34.769761 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:34.769761 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:34.769761 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:34.769761 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:34.769761 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:34.769761 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:34.769761 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:34.769761 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:34.769761 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:34.769761 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:34.769761 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:34.769761 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:34.769761 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:03:34.770675 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:03:34.770675 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:34.770675 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:34.770675 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 354/691] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:03:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:34.781969 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11511484-5567-4a5a-8cca-a0368013b091 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:34.900998 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11511484-5567-4a5a-8cca-a0368013b091 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:34.949819 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ea00f3-c90e-4116-979c-4beab5959663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:34.950465 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ea00f3-c90e-4116-979c-4beab5959663 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:34.950741 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ea00f3-c90e-4116-979c-4beab5959663 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:34.951009 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ea00f3-c90e-4116-979c-4beab5959663 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:34.951678 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ea00f3-c90e-4116-979c-4beab5959663 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:34.951982 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ea00f3-c90e-4116-979c-4beab5959663 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:34.953427 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ea00f3-c90e-4116-979c-4beab5959663 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:34.978048 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 338/692] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:03:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:34.985876 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1996571-a789-471b-a534-d07eaffb8667 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:35.144734 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1996571-a789-471b-a534-d07eaffb8667 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:35.170801 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:35.182788 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e733c0e5-6e58-4eae-afe6-c6b3d1544b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:35.183426 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e733c0e5-6e58-4eae-afe6-c6b3d1544b31 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:35.183699 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e733c0e5-6e58-4eae-afe6-c6b3d1544b31 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:35.183970 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e733c0e5-6e58-4eae-afe6-c6b3d1544b31 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:35.195514 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e733c0e5-6e58-4eae-afe6-c6b3d1544b31 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:35.196983 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e733c0e5-6e58-4eae-afe6-c6b3d1544b31 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:35.213588 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-e733c0e5-6e58-4eae-afe6-c6b3d1544b31 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:03:35.213588 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:35.213588 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:35.213588 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:35.213588 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:35.213588 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:35.213588 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:35.213588 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:35.213588 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:35.213588 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:35.213588 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:35.213588 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:35.213588 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:03:35.213588 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:03:35.213588 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:03:35.214607 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:03:35.214607 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:35.214607 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:35.214607 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:35.214607 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:35.214607 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:35.214607 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:35.214607 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:35.214607 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:35.214607 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:35.214607 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:35.214607 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:35.214607 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:35.214607 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:35.214607 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:35.214607 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:35.214607 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:35.215575 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:03:35.215575 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:03:35.215575 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:03:35.215575 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:03:35.215575 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:03:35.215575 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:03:35.215575 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:03:35.215575 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:35.215575 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 355/693] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 6 16:03:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:35.220377 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ea00f3-c90e-4116-979c-4beab5959663 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:35.227518 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:35.372539 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ea00f3-c90e-4116-979c-4beab5959663 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:35.405865 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c875f78-38fd-4f97-9dd1-280973080711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:35.406396 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c875f78-38fd-4f97-9dd1-280973080711 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:35.406667 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c875f78-38fd-4f97-9dd1-280973080711 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:35.406935 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c875f78-38fd-4f97-9dd1-280973080711 None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:35.407631 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c875f78-38fd-4f97-9dd1-280973080711 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:35.407931 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c875f78-38fd-4f97-9dd1-280973080711 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:35.409420 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c875f78-38fd-4f97-9dd1-280973080711 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:35.421170 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 339/694] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 6 16:03:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:35.429816 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e733c0e5-6e58-4eae-afe6-c6b3d1544b31 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:35.560346 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e733c0e5-6e58-4eae-afe6-c6b3d1544b31 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:35.612270 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef87e56-4071-4d5d-a527-b6b304ddaacc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:35.613111 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef87e56-4071-4d5d-a527-b6b304ddaacc None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:35.613395 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef87e56-4071-4d5d-a527-b6b304ddaacc None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:35.613672 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef87e56-4071-4d5d-a527-b6b304ddaacc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:35.614398 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef87e56-4071-4d5d-a527-b6b304ddaacc None admin] RBAC: Adding query filter params (role.id=31321627b02c4ff188f8fe1521ce3181, scope.project.id=128d210e8dad48ac921ed4cb1d444ceb, user.id=312026c080b64aff9dd483d3591288a6) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:35.614702 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef87e56-4071-4d5d-a527-b6b304ddaacc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:35.616161 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef87e56-4071-4d5d-a527-b6b304ddaacc None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:35.626188 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 356/695] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 6 16:03:35 2022] GET /identity/v3/role_assignments?user.id=312026c080b64aff9dd483d3591288a6&scope.project.id=128d210e8dad48ac921ed4cb1d444ceb&role.id=31321627b02c4ff188f8fe1521ce3181 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:35.979537 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b4101f-7763-4211-aaa8-48a541abaeb8 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:35.980472 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b4101f-7763-4211-aaa8-48a541abaeb8 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:35.981010 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b4101f-7763-4211-aaa8-48a541abaeb8 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:35.982846 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 340/696] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:03:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:03:35.988089 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e08e7c-6e34-411b-8a26-2132ce3dd82c None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:35.988376 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e08e7c-6e34-411b-8a26-2132ce3dd82c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:35.988648 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e08e7c-6e34-411b-8a26-2132ce3dd82c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:36.011344 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-b5e08e7c-6e34-411b-8a26-2132ce3dd82c None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:03:36.012541 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b5e08e7c-6e34-411b-8a26-2132ce3dd82c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:03:36.080780 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:36.106620 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 357/697] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:03:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:03:36.219146 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabde740-cfe8-4c6f-897f-151ce4a2f1ee None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:36.219801 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabde740-cfe8-4c6f-897f-151ce4a2f1ee None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:36.220277 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabde740-cfe8-4c6f-897f-151ce4a2f1ee None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:36.222530 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 341/698] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:03:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:03:36.228201 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecd9bad-cf6b-4ba0-a347-a84896e218b6 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:36.228490 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecd9bad-cf6b-4ba0-a347-a84896e218b6 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:36.228778 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecd9bad-cf6b-4ba0-a347-a84896e218b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:36.252038 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-0ecd9bad-cf6b-4ba0-a347-a84896e218b6 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:03:36.253187 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0ecd9bad-cf6b-4ba0-a347-a84896e218b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:03:36.343722 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 358/699] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:03:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:03:36.461437 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabde740-cfe8-4c6f-897f-151ce4a2f1ee None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:36.544994 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:36.599392 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabde740-cfe8-4c6f-897f-151ce4a2f1ee None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:36.640057 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7516ca87-97b0-4fb4-a024-703133b87992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:36.641118 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7516ca87-97b0-4fb4-a024-703133b87992 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:36.641603 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7516ca87-97b0-4fb4-a024-703133b87992 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:36.642087 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7516ca87-97b0-4fb4-a024-703133b87992 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:36.654155 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7516ca87-97b0-4fb4-a024-703133b87992 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:03:36.664438 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-7516ca87-97b0-4fb4-a024-703133b87992 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:03:36.664438 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:36.664438 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:36.664438 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:36.664438 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:36.664438 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:36.664438 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:36.664438 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:36.664438 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:36.664438 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:36.664438 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:36.664438 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:36.664438 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:03:36.664438 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:03:36.664438 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:03:36.664438 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:03:36.665441 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:03:36.665441 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:03:36.665441 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:03:36.665441 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:03:36.665441 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:36.665441 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:36.665441 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:36.665441 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:36.665441 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:36.665441 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:36.665441 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:36.665441 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:36.665441 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:36.665441 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:36.665441 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:36.665441 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:36.666305 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:36.666305 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:36.666305 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:36.666305 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:36.666305 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:03:36.666305 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:03:36.666305 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:03:36.666305 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:03:36.666305 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:03:36.666305 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:03:36.666305 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:03:36.666305 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:36.668961 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 342/700] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 6 16:03:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:36.674612 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecd9bad-cf6b-4ba0-a347-a84896e218b6 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:36.700700 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7516ca87-97b0-4fb4-a024-703133b87992 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:36.805418 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecd9bad-cf6b-4ba0-a347-a84896e218b6 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:36.808414 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7516ca87-97b0-4fb4-a024-703133b87992 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:36.821154 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:36.832027 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:36.843846 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afde167a-5cff-4795-a722-13ad36ce5d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:36.844483 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde167a-5cff-4795-a722-13ad36ce5d71 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:36.844802 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde167a-5cff-4795-a722-13ad36ce5d71 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:36.845060 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde167a-5cff-4795-a722-13ad36ce5d71 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:36.845703 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afde167a-5cff-4795-a722-13ad36ce5d71 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:36.847131 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afde167a-5cff-4795-a722-13ad36ce5d71 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:36.851345 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999b0a24-5662-4e3b-93ec-feb2be8bfc83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:36.852194 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999b0a24-5662-4e3b-93ec-feb2be8bfc83 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:36.852685 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999b0a24-5662-4e3b-93ec-feb2be8bfc83 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:36.853187 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999b0a24-5662-4e3b-93ec-feb2be8bfc83 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:36.854049 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999b0a24-5662-4e3b-93ec-feb2be8bfc83 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:36.854535 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999b0a24-5662-4e3b-93ec-feb2be8bfc83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:36.856090 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999b0a24-5662-4e3b-93ec-feb2be8bfc83 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:36.859000 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-afde167a-5cff-4795-a722-13ad36ce5d71 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 16:03:36.859000 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:36.859000 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 16:03:36.859000 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 16:03:36.859000 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 16:03:36.859000 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 16:03:36.859000 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 16:03:36.859000 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:03:36.859000 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 16:03:36.859000 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:36.859000 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 16:03:36.859000 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:36.859000 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:36.859000 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:36.859000 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:36.859000 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:36.859000 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:36.860019 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:36.860019 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:36.860019 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:36.860019 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:36.860019 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:36.860019 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:36.860019 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 16:03:36.860019 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 16:03:36.860019 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 16:03:36.860019 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 16:03:36.860019 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:36.860019 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:36.860019 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:36.860019 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:36.860019 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:36.860019 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:36.860891 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:36.860891 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:36.860891 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:36.860891 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:36.860891 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:36.860891 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:36.860891 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:36.860891 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:36.860891 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:36.860891 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:36.860891 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 16:03:36.860891 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:03:36.860891 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 16:03:36.860891 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:36.860891 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 343/701] 10.222.0.43 () {56 vars in 1129 bytes} [Tue Sep 6 16:03:36 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:36.861704 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b041917-82ed-42b4-9dd2-b5754b5ded5f None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:36.862092 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b041917-82ed-42b4-9dd2-b5754b5ded5f None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:36.862198 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b041917-82ed-42b4-9dd2-b5754b5ded5f None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:36.863650 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 344/702] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:03:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:03:36.868298 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b041917-82ed-42b4-9dd2-b5754b5ded5f None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:36.869834 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 359/703] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 6 16:03:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:36.871356 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf016a6-a894-4a17-9f68-1fea87de038a None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:36.871935 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf016a6-a894-4a17-9f68-1fea87de038a None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:36.872413 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf016a6-a894-4a17-9f68-1fea87de038a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:36.901648 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-baf016a6-a894-4a17-9f68-1fea87de038a None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:03:36.903236 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-baf016a6-a894-4a17-9f68-1fea87de038a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:03:36.983598 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 360/704] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:03:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:03:36.984799 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf016a6-a894-4a17-9f68-1fea87de038a None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:37.018840 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b041917-82ed-42b4-9dd2-b5754b5ded5f None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:37.087732 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e9da42-4e34-4fd1-9c2c-168a8829633a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:37.088734 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e9da42-4e34-4fd1-9c2c-168a8829633a None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:37.089327 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e9da42-4e34-4fd1-9c2c-168a8829633a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:37.089907 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e9da42-4e34-4fd1-9c2c-168a8829633a None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:37.090757 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e9da42-4e34-4fd1-9c2c-168a8829633a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:37.091306 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e9da42-4e34-4fd1-9c2c-168a8829633a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:37.093124 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e9da42-4e34-4fd1-9c2c-168a8829633a None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:37.106733 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 345/705] 10.222.0.43 () {56 vars in 1124 bytes} [Tue Sep 6 16:03:36 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:37.112184 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e9da42-4e34-4fd1-9c2c-168a8829633a None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:37.139359 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf016a6-a894-4a17-9f68-1fea87de038a None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:37.144835 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:37.198396 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9142a844-a2ac-4d69-bc01-7685f62bc9c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:37.199039 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9142a844-a2ac-4d69-bc01-7685f62bc9c5 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:37.199307 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9142a844-a2ac-4d69-bc01-7685f62bc9c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:37.199672 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9142a844-a2ac-4d69-bc01-7685f62bc9c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:37.200804 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:37.213093 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9142a844-a2ac-4d69-bc01-7685f62bc9c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:37.214537 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9142a844-a2ac-4d69-bc01-7685f62bc9c5 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:37.230965 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-9142a844-a2ac-4d69-bc01-7685f62bc9c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:37.230965 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:37.230965 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:37.230965 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:37.230965 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:37.230965 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:37.230965 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:37.230965 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:37.230965 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:37.230965 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:37.230965 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:37.230965 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:37.230965 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:03:37.230965 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:03:37.230965 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:03:37.230965 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:03:37.231988 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:37.231988 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:37.231988 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:37.231988 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:37.231988 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:37.231988 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:37.231988 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:37.231988 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:37.231988 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:37.231988 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:37.231988 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:37.231988 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:37.231988 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:37.231988 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:37.231988 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:37.231988 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:37.231988 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:03:37.232922 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:03:37.232922 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:37.232922 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:37.232922 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 361/706] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:03:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:37.233726 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6e55aa-cb68-4073-a5a3-94fe0af90590 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:37.234005 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6e55aa-cb68-4073-a5a3-94fe0af90590 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:37.234272 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6e55aa-cb68-4073-a5a3-94fe0af90590 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:37.240169 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 362/707] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:03:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:03:37.241264 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6e55aa-cb68-4073-a5a3-94fe0af90590 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:37.264946 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e9da42-4e34-4fd1-9c2c-168a8829633a None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:37.308078 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e0be6f-cf50-4660-92e7-82ee75765201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:37.309334 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0be6f-cf50-4660-92e7-82ee75765201 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:37.309835 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0be6f-cf50-4660-92e7-82ee75765201 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:37.310302 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0be6f-cf50-4660-92e7-82ee75765201 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:37.311180 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e0be6f-cf50-4660-92e7-82ee75765201 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:37.311691 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e0be6f-cf50-4660-92e7-82ee75765201 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:37.313406 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e0be6f-cf50-4660-92e7-82ee75765201 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:37.320782 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 346/708] 10.222.0.43 () {56 vars in 1124 bytes} [Tue Sep 6 16:03:37 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:37.322449 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff7826d-5358-42ef-9de7-f72645d4cc1a None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:37.322951 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff7826d-5358-42ef-9de7-f72645d4cc1a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:37.323420 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff7826d-5358-42ef-9de7-f72645d4cc1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:37.345816 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-0ff7826d-5358-42ef-9de7-f72645d4cc1a None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:03:37.347654 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0ff7826d-5358-42ef-9de7-f72645d4cc1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:03:37.441002 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 347/709] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:03:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:03:37.441002 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff7826d-5358-42ef-9de7-f72645d4cc1a None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:37.491986 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:37.539135 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6e55aa-cb68-4073-a5a3-94fe0af90590 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:37.552785 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff7826d-5358-42ef-9de7-f72645d4cc1a None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:37.589445 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d678177e-46d7-4600-ab8e-c22ca88daad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:37.590060 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d678177e-46d7-4600-ab8e-c22ca88daad2 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:37.590338 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d678177e-46d7-4600-ab8e-c22ca88daad2 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:37.590603 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d678177e-46d7-4600-ab8e-c22ca88daad2 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:37.591245 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d678177e-46d7-4600-ab8e-c22ca88daad2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:37.591539 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d678177e-46d7-4600-ab8e-c22ca88daad2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:37.592945 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d678177e-46d7-4600-ab8e-c22ca88daad2 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:37.600593 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27b328b-cb15-4fb8-8dd6-3ee6b6c6ebaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:37.601561 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27b328b-cb15-4fb8-8dd6-3ee6b6c6ebaf None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:37.602080 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27b328b-cb15-4fb8-8dd6-3ee6b6c6ebaf None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:37.602542 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27b328b-cb15-4fb8-8dd6-3ee6b6c6ebaf None admin] PATH_INFO: `/v3/services/network` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:37.603426 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27b328b-cb15-4fb8-8dd6-3ee6b6c6ebaf None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:37.605224 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27b328b-cb15-4fb8-8dd6-3ee6b6c6ebaf None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:37.612322 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 363/710] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:03:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:37.619872 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d678177e-46d7-4600-ab8e-c22ca88daad2 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:37.626540 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-a27b328b-cb15-4fb8-8dd6-3ee6b6c6ebaf None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 16:03:37.626540 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:37.626540 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 16:03:37.626540 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 16:03:37.626540 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 16:03:37.626540 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 16:03:37.626540 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 16:03:37.626540 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:03:37.626540 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 16:03:37.626540 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:37.626540 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 16:03:37.626540 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:37.626540 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:37.626540 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:37.626540 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:37.626540 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:37.626540 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:37.627566 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:37.627566 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:37.627566 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:37.627566 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:37.627566 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:37.627566 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:37.627566 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 16:03:37.627566 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 16:03:37.627566 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 16:03:37.627566 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 16:03:37.627566 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:37.627566 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:37.627566 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:37.627566 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:37.627566 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:37.627566 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:37.628449 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:37.628449 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:37.628449 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:37.628449 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:37.628449 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:37.628449 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:37.628449 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:37.628449 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:37.628449 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:37.628449 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:37.628449 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 16:03:37.628449 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:03:37.628449 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 16:03:37.628449 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:37.634400 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 348/711] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 6 16:03:37 2022] GET /identity/v3/services/network => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:37.643017 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27b328b-cb15-4fb8-8dd6-3ee6b6c6ebaf None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:37.667886 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:37.807136 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d678177e-46d7-4600-ab8e-c22ca88daad2 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:37.812457 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27b328b-cb15-4fb8-8dd6-3ee6b6c6ebaf None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:37.824590 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:37.850854 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41c6740-a30a-4fb5-831c-b25434c819d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:37.851418 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c6740-a30a-4fb5-831c-b25434c819d0 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:37.851697 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c6740-a30a-4fb5-831c-b25434c819d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:37.851969 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c6740-a30a-4fb5-831c-b25434c819d0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:37.859119 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7a723f-79d2-4a0f-94f6-942a4debeb66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:37.860285 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a723f-79d2-4a0f-94f6-942a4debeb66 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:37.860462 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41c6740-a30a-4fb5-831c-b25434c819d0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:37.861008 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a723f-79d2-4a0f-94f6-942a4debeb66 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:37.861486 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a723f-79d2-4a0f-94f6-942a4debeb66 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:37.861991 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41c6740-a30a-4fb5-831c-b25434c819d0 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:37.862629 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7a723f-79d2-4a0f-94f6-942a4debeb66 None admin] RBAC: Adding query filter params (name=network) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:37.863167 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7a723f-79d2-4a0f-94f6-942a4debeb66 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:37.865721 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7a723f-79d2-4a0f-94f6-942a4debeb66 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:37.871303 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 349/712] 10.222.0.43 () {56 vars in 1112 bytes} [Tue Sep 6 16:03:37 2022] GET /identity/v3/services?name=network => generated 124 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:37.875777 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-e41c6740-a30a-4fb5-831c-b25434c819d0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 16:03:37.875777 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:37.875777 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:37.875777 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:37.875777 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:37.875777 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:37.875777 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:37.875777 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:37.875777 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:37.875777 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:37.875777 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:37.875777 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:37.875777 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:03:37.875777 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:03:37.875777 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:03:37.875777 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:03:37.876708 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:37.876708 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:37.876708 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:03:37.876708 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:03:37.876708 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:03:37.876708 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:03:37.876708 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:37.876708 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:37.876708 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:37.876708 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:37.876708 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:37.876708 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:37.876708 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:37.876708 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:37.876708 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:37.876708 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:37.876708 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:37.876708 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:37.877707 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:37.877707 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:37.877707 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:03:37.877707 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:03:37.877707 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:03:37.877707 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:03:37.877707 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:03:37.877707 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:03:37.877707 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 16:03:37.877707 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:37.877707 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 364/713] 10.222.0.43 () {56 vars in 1093 bytes} [Tue Sep 6 16:03:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:37.881411 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41c6740-a30a-4fb5-831c-b25434c819d0 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:37.882512 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7a723f-79d2-4a0f-94f6-942a4debeb66 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:37.977888 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:37.996008 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7a723f-79d2-4a0f-94f6-942a4debeb66 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:38.003988 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41c6740-a30a-4fb5-831c-b25434c819d0 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:38.031007 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aadebe-9dd8-483e-8b29-5c064e7f043e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:38.031551 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aadebe-9dd8-483e-8b29-5c064e7f043e None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:38.031834 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aadebe-9dd8-483e-8b29-5c064e7f043e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:38.032109 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aadebe-9dd8-483e-8b29-5c064e7f043e None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:38.032735 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aadebe-9dd8-483e-8b29-5c064e7f043e None admin] RBAC: Adding query filter params (type=network) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:38.033065 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aadebe-9dd8-483e-8b29-5c064e7f043e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:38.034520 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aadebe-9dd8-483e-8b29-5c064e7f043e None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:38.044636 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 350/714] 10.222.0.43 () {56 vars in 1112 bytes} [Tue Sep 6 16:03:37 2022] GET /identity/v3/services?type=network => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:38.044636 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aadebe-9dd8-483e-8b29-5c064e7f043e None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:38.052861 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547a8fde-049b-45cc-a8e4-11a6de7a2f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:38.053980 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547a8fde-049b-45cc-a8e4-11a6de7a2f75 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:38.054470 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547a8fde-049b-45cc-a8e4-11a6de7a2f75 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:38.054928 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547a8fde-049b-45cc-a8e4-11a6de7a2f75 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:38.079344 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:38.087108 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547a8fde-049b-45cc-a8e4-11a6de7a2f75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:03:38.092714 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-547a8fde-049b-45cc-a8e4-11a6de7a2f75 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:03:38.092714 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:38.092714 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:38.092714 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:38.092714 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:38.092714 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:38.092714 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:38.092714 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:38.092714 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:38.092714 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:38.092714 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:38.092714 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:38.092714 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:03:38.092714 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:03:38.092714 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:03:38.092714 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:03:38.093664 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:03:38.093664 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:03:38.093664 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:03:38.093664 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:03:38.093664 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:38.093664 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:38.093664 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:38.093664 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:38.093664 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:38.093664 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:38.093664 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:38.093664 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:38.093664 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:38.093664 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:38.093664 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:38.093664 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:38.094529 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:38.094529 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:38.094529 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:38.094529 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:38.094529 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:03:38.094529 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:03:38.094529 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:03:38.094529 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:03:38.094529 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:03:38.094529 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:03:38.094529 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:03:38.094529 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:38.106043 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 365/715] 10.222.0.43 () {56 vars in 1102 bytes} [Tue Sep 6 16:03:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:38.107984 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69423702-9988-40a9-9c3d-d4307a546611 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:38.108505 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69423702-9988-40a9-9c3d-d4307a546611 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:38.109049 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69423702-9988-40a9-9c3d-d4307a546611 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:38.112432 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 366/716] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:03:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:03:38.113680 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69423702-9988-40a9-9c3d-d4307a546611 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:38.152525 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aadebe-9dd8-483e-8b29-5c064e7f043e None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:38.189924 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817818b6-b9b9-428b-8043-30bdf07fd740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:38.190614 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817818b6-b9b9-428b-8043-30bdf07fd740 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:38.190980 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817818b6-b9b9-428b-8043-30bdf07fd740 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:38.191255 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817818b6-b9b9-428b-8043-30bdf07fd740 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:38.192014 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817818b6-b9b9-428b-8043-30bdf07fd740 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:38.192320 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817818b6-b9b9-428b-8043-30bdf07fd740 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:38.193912 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817818b6-b9b9-428b-8043-30bdf07fd740 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:38.218709 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 351/717] 10.222.0.43 () {56 vars in 1133 bytes} [Tue Sep 6 16:03:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:38.220457 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bff2ec2-36d1-409e-b91c-6519832bb2a4 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:38.221728 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bff2ec2-36d1-409e-b91c-6519832bb2a4 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:38.224028 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bff2ec2-36d1-409e-b91c-6519832bb2a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:38.245431 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-1bff2ec2-36d1-409e-b91c-6519832bb2a4 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:03:38.246823 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1bff2ec2-36d1-409e-b91c-6519832bb2a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:03:38.299264 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69423702-9988-40a9-9c3d-d4307a546611 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:38.364116 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c6fc29-c3b4-445b-a5be-57d6df23e3da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:38.364892 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c6fc29-c3b4-445b-a5be-57d6df23e3da None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:38.365268 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c6fc29-c3b4-445b-a5be-57d6df23e3da None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:38.365636 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c6fc29-c3b4-445b-a5be-57d6df23e3da None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:38.367669 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c6fc29-c3b4-445b-a5be-57d6df23e3da None admin] RBAC: Adding query filter params (name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:38.368958 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c6fc29-c3b4-445b-a5be-57d6df23e3da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:38.372160 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c6fc29-c3b4-445b-a5be-57d6df23e3da None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:38.378927 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:38.391448 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 367/718] 10.222.0.43 () {56 vars in 1103 bytes} [Tue Sep 6 16:03:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:38.393422 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c6fc29-c3b4-445b-a5be-57d6df23e3da None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:38.400476 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 352/719] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:03:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:03:38.401500 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bff2ec2-36d1-409e-b91c-6519832bb2a4 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:38.453164 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:38.565710 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bff2ec2-36d1-409e-b91c-6519832bb2a4 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:38.626035 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c6fc29-c3b4-445b-a5be-57d6df23e3da None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:38.686664 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b261664e-b41e-45a1-9619-14b1de5c7f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:38.687691 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b261664e-b41e-45a1-9619-14b1de5c7f1d None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:38.688195 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b261664e-b41e-45a1-9619-14b1de5c7f1d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:38.688661 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b261664e-b41e-45a1-9619-14b1de5c7f1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:38.701497 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64745e77-5189-4411-9fd1-e70cb5ec43d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:38.702155 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64745e77-5189-4411-9fd1-e70cb5ec43d1 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:38.702459 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64745e77-5189-4411-9fd1-e70cb5ec43d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:38.702702 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64745e77-5189-4411-9fd1-e70cb5ec43d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:38.728379 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64745e77-5189-4411-9fd1-e70cb5ec43d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:38.730150 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64745e77-5189-4411-9fd1-e70cb5ec43d1 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:38.753017 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-64745e77-5189-4411-9fd1-e70cb5ec43d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:38.753017 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:38.753017 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:38.753017 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:38.753017 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:38.753017 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:38.753017 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:38.753017 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:38.753017 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:38.753017 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:38.753017 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:38.753017 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:38.753017 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:03:38.753017 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:03:38.753017 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:03:38.753017 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:03:38.754135 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:38.754135 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:38.754135 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:38.754135 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:38.754135 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:38.754135 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:38.754135 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:38.754135 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:38.754135 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:38.754135 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:38.754135 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:38.754135 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:38.754135 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:38.754135 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:38.754135 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:38.754135 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:38.754135 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:03:38.755047 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:03:38.755047 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:38.755047 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:38.755047 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 353/720] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:03:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:38.755283 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64745e77-5189-4411-9fd1-e70cb5ec43d1 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:38.785248 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:38.790803 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b261664e-b41e-45a1-9619-14b1de5c7f1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:38.793056 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b261664e-b41e-45a1-9619-14b1de5c7f1d None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:38.873756 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:38.902907 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64745e77-5189-4411-9fd1-e70cb5ec43d1 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:38.938408 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d727d95-f4c2-4a7a-8276-3b75a16a5bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:38.939388 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d727d95-f4c2-4a7a-8276-3b75a16a5bb9 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:38.939862 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d727d95-f4c2-4a7a-8276-3b75a16a5bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:38.940326 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d727d95-f4c2-4a7a-8276-3b75a16a5bb9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:38.947471 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 368/721] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 6 16:03:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 551 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 16:03:38.947471 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b261664e-b41e-45a1-9619-14b1de5c7f1d None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:38.958374 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d727d95-f4c2-4a7a-8276-3b75a16a5bb9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:03:38.964162 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-3d727d95-f4c2-4a7a-8276-3b75a16a5bb9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:03:38.964162 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:38.964162 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:38.964162 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:38.964162 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:38.964162 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:38.964162 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:38.964162 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:38.964162 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:38.964162 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:38.964162 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:38.964162 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:38.964162 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:03:38.964162 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:03:38.964162 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:03:38.964162 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:03:38.965125 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:03:38.965125 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:03:38.965125 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:03:38.965125 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:03:38.965125 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:38.965125 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:38.965125 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:38.965125 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:38.965125 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:38.965125 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:38.965125 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:38.965125 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:38.965125 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:38.965125 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:38.965125 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:38.965125 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:38.965994 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:38.965994 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:38.965994 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:38.965994 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:38.965994 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:03:38.965994 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:03:38.965994 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:03:38.965994 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:03:38.965994 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:03:38.965994 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:03:38.965994 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:03:38.965994 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:38.968268 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 354/722] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 6 16:03:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:38.969366 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d727d95-f4c2-4a7a-8276-3b75a16a5bb9 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:39.051656 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:39.057808 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b261664e-b41e-45a1-9619-14b1de5c7f1d None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:39.096558 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bcf63e-99ee-48d6-bdb2-43aa38c54cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:39.097605 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bcf63e-99ee-48d6-bdb2-43aa38c54cb1 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:39.098083 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bcf63e-99ee-48d6-bdb2-43aa38c54cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:39.098577 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bcf63e-99ee-48d6-bdb2-43aa38c54cb1 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:39.099433 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bcf63e-99ee-48d6-bdb2-43aa38c54cb1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:39.099984 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bcf63e-99ee-48d6-bdb2-43aa38c54cb1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:39.101651 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bcf63e-99ee-48d6-bdb2-43aa38c54cb1 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:39.111399 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d727d95-f4c2-4a7a-8276-3b75a16a5bb9 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:39.115000 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 369/723] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:03:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:39.116131 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bcf63e-99ee-48d6-bdb2-43aa38c54cb1 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:39.148313 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecb4b41-a314-43b5-abe8-955e443a5e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:39.149524 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb4b41-a314-43b5-abe8-955e443a5e4e None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:39.150023 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb4b41-a314-43b5-abe8-955e443a5e4e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:39.150568 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb4b41-a314-43b5-abe8-955e443a5e4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:39.154050 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:39.163554 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecb4b41-a314-43b5-abe8-955e443a5e4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:39.165374 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecb4b41-a314-43b5-abe8-955e443a5e4e None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:39.180704 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-eecb4b41-a314-43b5-abe8-955e443a5e4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:39.180704 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:39.180704 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:39.180704 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:39.180704 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:39.180704 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:39.180704 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:39.180704 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:39.180704 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:39.180704 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:39.180704 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:39.180704 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:39.180704 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:03:39.180704 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:03:39.180704 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:03:39.180704 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:03:39.181809 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:39.181809 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:39.181809 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:39.181809 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:39.181809 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:39.181809 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:39.181809 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:39.181809 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:39.181809 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:39.181809 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:39.181809 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:39.181809 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:39.181809 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:39.181809 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:39.181809 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:39.181809 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:39.181809 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:03:39.182708 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:03:39.182708 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:39.182708 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:39.189165 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 355/724] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:03:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:39.190240 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecb4b41-a314-43b5-abe8-955e443a5e4e None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:39.241135 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bcf63e-99ee-48d6-bdb2-43aa38c54cb1 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:39.291791 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb71eb49-9aa7-4643-a86e-5884ada6bc0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:39.292420 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb71eb49-9aa7-4643-a86e-5884ada6bc0c None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:39.292700 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb71eb49-9aa7-4643-a86e-5884ada6bc0c None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:39.293015 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb71eb49-9aa7-4643-a86e-5884ada6bc0c None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:39.294150 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb71eb49-9aa7-4643-a86e-5884ada6bc0c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:39.294460 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb71eb49-9aa7-4643-a86e-5884ada6bc0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:39.296350 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb71eb49-9aa7-4643-a86e-5884ada6bc0c None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:39.312827 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecb4b41-a314-43b5-abe8-955e443a5e4e None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:39.313394 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 370/725] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 6 16:03:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:39.314452 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb71eb49-9aa7-4643-a86e-5884ada6bc0c None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:39.345153 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:39.350585 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debfee4e-5e32-46b4-b4d2-fe6863b64994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:39.351478 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debfee4e-5e32-46b4-b4d2-fe6863b64994 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:39.351951 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debfee4e-5e32-46b4-b4d2-fe6863b64994 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:39.352469 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debfee4e-5e32-46b4-b4d2-fe6863b64994 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:39.360433 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debfee4e-5e32-46b4-b4d2-fe6863b64994 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:39.362180 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debfee4e-5e32-46b4-b4d2-fe6863b64994 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:39.374097 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-debfee4e-5e32-46b4-b4d2-fe6863b64994 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 16:03:39.374097 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:39.374097 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:39.374097 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:39.374097 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:39.374097 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:39.374097 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:39.374097 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:39.374097 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:39.374097 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:39.374097 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:39.374097 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:39.374097 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:03:39.374097 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:03:39.374097 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:03:39.374097 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:03:39.375045 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:39.375045 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:39.375045 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:03:39.375045 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:03:39.375045 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:03:39.375045 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:03:39.375045 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:39.375045 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:39.375045 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:39.375045 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:39.375045 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:39.375045 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:39.375045 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:39.375045 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:39.375045 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:39.375045 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:39.375045 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:39.375045 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:39.375991 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:39.375991 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:39.375991 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:03:39.375991 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:03:39.375991 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:03:39.375991 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:03:39.375991 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:03:39.375991 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:03:39.375991 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 16:03:39.375991 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:39.378498 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 356/726] 10.222.0.43 () {56 vars in 1117 bytes} [Tue Sep 6 16:03:39 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:39.379549 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debfee4e-5e32-46b4-b4d2-fe6863b64994 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:39.417329 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb71eb49-9aa7-4643-a86e-5884ada6bc0c None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:39.434469 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:39.456177 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc7b8ff-6fc1-45f4-9818-d129a733ddaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:39.456922 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc7b8ff-6fc1-45f4-9818-d129a733ddaf None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:39.457241 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc7b8ff-6fc1-45f4-9818-d129a733ddaf None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:39.457581 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc7b8ff-6fc1-45f4-9818-d129a733ddaf None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:39.458238 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc7b8ff-6fc1-45f4-9818-d129a733ddaf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:39.458642 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc7b8ff-6fc1-45f4-9818-d129a733ddaf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:39.460260 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc7b8ff-6fc1-45f4-9818-d129a733ddaf None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:39.477100 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 371/727] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:03:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:39.477100 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc7b8ff-6fc1-45f4-9818-d129a733ddaf None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:39.496473 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debfee4e-5e32-46b4-b4d2-fe6863b64994 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:39.533085 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cafb520-b74e-4993-80d8-c6896870a32e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:39.534137 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cafb520-b74e-4993-80d8-c6896870a32e None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:39.534695 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cafb520-b74e-4993-80d8-c6896870a32e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:39.535257 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cafb520-b74e-4993-80d8-c6896870a32e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:39.547098 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cafb520-b74e-4993-80d8-c6896870a32e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:39.548996 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cafb520-b74e-4993-80d8-c6896870a32e None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:39.559577 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:39.566063 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-8cafb520-b74e-4993-80d8-c6896870a32e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:39.566063 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:39.566063 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:39.566063 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:39.566063 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:39.566063 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:39.566063 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:39.566063 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:39.566063 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:39.566063 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:39.566063 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:39.566063 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:39.566063 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:03:39.566063 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:03:39.566063 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:03:39.566063 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:03:39.567344 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:39.567344 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:39.567344 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:39.567344 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:39.567344 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:39.567344 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:39.567344 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:39.567344 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:39.567344 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:39.567344 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:39.567344 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:39.567344 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:39.567344 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:39.567344 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:39.567344 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:39.567344 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:39.567344 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:03:39.568270 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:03:39.568270 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:39.568270 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:39.568270 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 357/728] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:03:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:39.568270 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cafb520-b74e-4993-80d8-c6896870a32e None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:39.583127 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc7b8ff-6fc1-45f4-9818-d129a733ddaf None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:39.619854 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9da500-5322-4a24-a819-304c46a6c986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:39.620805 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9da500-5322-4a24-a819-304c46a6c986 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:39.621255 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9da500-5322-4a24-a819-304c46a6c986 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:39.621728 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9da500-5322-4a24-a819-304c46a6c986 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:39.622646 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9da500-5322-4a24-a819-304c46a6c986 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:39.623130 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9da500-5322-4a24-a819-304c46a6c986 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:39.624704 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9da500-5322-4a24-a819-304c46a6c986 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:39.642815 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 372/729] 10.222.0.43 () {56 vars in 1149 bytes} [Tue Sep 6 16:03:39 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:39.644288 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826f6498-0fb4-439b-82eb-2309d89fbf87 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:39.644734 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826f6498-0fb4-439b-82eb-2309d89fbf87 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:39.645225 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826f6498-0fb4-439b-82eb-2309d89fbf87 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:39.646845 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 373/730] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:03:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:03:39.647991 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826f6498-0fb4-439b-82eb-2309d89fbf87 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:39.686597 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:39.690041 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cafb520-b74e-4993-80d8-c6896870a32e None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:39.734659 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabcd9e4-2cd0-4b12-ba2c-de2084dc5c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:39.735300 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabcd9e4-2cd0-4b12-ba2c-de2084dc5c42 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:39.735569 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabcd9e4-2cd0-4b12-ba2c-de2084dc5c42 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:39.736197 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabcd9e4-2cd0-4b12-ba2c-de2084dc5c42 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:39.743707 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabcd9e4-2cd0-4b12-ba2c-de2084dc5c42 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:39.745212 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabcd9e4-2cd0-4b12-ba2c-de2084dc5c42 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:39.758092 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-eabcd9e4-2cd0-4b12-ba2c-de2084dc5c42 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:03:39.758092 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:39.758092 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:39.758092 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:39.758092 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:39.758092 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:39.758092 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:39.758092 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:39.758092 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:39.758092 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:39.758092 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:39.758092 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:39.758092 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:03:39.758092 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:03:39.758092 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:03:39.759071 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:03:39.759071 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:39.759071 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:39.759071 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:39.759071 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:39.759071 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:39.759071 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:39.759071 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:39.759071 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:39.759071 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:39.759071 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:39.759071 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:39.759071 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:39.759071 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:39.759071 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:39.759071 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:39.759071 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:39.759965 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:03:39.759965 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:03:39.759965 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:03:39.759965 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:03:39.759965 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:03:39.759965 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:03:39.759965 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:03:39.759965 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:39.759965 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 358/731] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 6 16:03:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:39.760443 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabcd9e4-2cd0-4b12-ba2c-de2084dc5c42 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:39.768906 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826f6498-0fb4-439b-82eb-2309d89fbf87 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:39.803349 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09b4c3d-8a20-41a4-97a9-e8e062df6a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:39.804300 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09b4c3d-8a20-41a4-97a9-e8e062df6a0e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:39.804789 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09b4c3d-8a20-41a4-97a9-e8e062df6a0e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:39.805267 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09b4c3d-8a20-41a4-97a9-e8e062df6a0e None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:39.806075 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09b4c3d-8a20-41a4-97a9-e8e062df6a0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:39.806552 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09b4c3d-8a20-41a4-97a9-e8e062df6a0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:39.808089 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09b4c3d-8a20-41a4-97a9-e8e062df6a0e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:39.815296 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:39.820907 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 374/732] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:03:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:39.822434 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4a6830-5b7f-4603-8be0-06b92b3758cb None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:39.822796 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4a6830-5b7f-4603-8be0-06b92b3758cb None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:39.823125 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4a6830-5b7f-4603-8be0-06b92b3758cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:39.841307 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-3c4a6830-5b7f-4603-8be0-06b92b3758cb None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:03:39.842493 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-3c4a6830-5b7f-4603-8be0-06b92b3758cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:03:39.873188 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabcd9e4-2cd0-4b12-ba2c-de2084dc5c42 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:39.907933 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e444856a-c4e7-454e-9fa7-5cbbfc16b56a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:39.908544 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e444856a-c4e7-454e-9fa7-5cbbfc16b56a None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:39.908862 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e444856a-c4e7-454e-9fa7-5cbbfc16b56a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:39.909128 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e444856a-c4e7-454e-9fa7-5cbbfc16b56a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:39.916890 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 375/733] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:03:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:03:39.918544 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4a6830-5b7f-4603-8be0-06b92b3758cb None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:39.938724 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:39.981789 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e444856a-c4e7-454e-9fa7-5cbbfc16b56a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:39.983280 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e444856a-c4e7-454e-9fa7-5cbbfc16b56a None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:40.055832 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4a6830-5b7f-4603-8be0-06b92b3758cb None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:40.085230 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:40.095647 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399626bb-007e-40ef-996c-8fab971cfcb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:40.096534 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399626bb-007e-40ef-996c-8fab971cfcb2 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:40.096873 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399626bb-007e-40ef-996c-8fab971cfcb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:40.097311 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399626bb-007e-40ef-996c-8fab971cfcb2 None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:40.098328 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399626bb-007e-40ef-996c-8fab971cfcb2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:40.098902 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399626bb-007e-40ef-996c-8fab971cfcb2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:40.100356 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399626bb-007e-40ef-996c-8fab971cfcb2 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:40.101435 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 359/734] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 6 16:03:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 16:03:40.102650 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e444856a-c4e7-454e-9fa7-5cbbfc16b56a None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:40.115521 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 376/735] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 6 16:03:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:40.116970 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afcf35a-f7d8-4f90-a9f0-2bec6e0cd684 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:40.117320 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afcf35a-f7d8-4f90-a9f0-2bec6e0cd684 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:40.117662 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afcf35a-f7d8-4f90-a9f0-2bec6e0cd684 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:40.120126 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 377/736] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:03:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:03:40.120126 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afcf35a-f7d8-4f90-a9f0-2bec6e0cd684 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:40.175701 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:40.218003 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e444856a-c4e7-454e-9fa7-5cbbfc16b56a None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:40.223286 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afcf35a-f7d8-4f90-a9f0-2bec6e0cd684 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:40.253024 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4357b0e2-5d55-4129-84e7-a6ea39612830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:40.253716 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4357b0e2-5d55-4129-84e7-a6ea39612830 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:40.253974 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4357b0e2-5d55-4129-84e7-a6ea39612830 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:40.254295 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4357b0e2-5d55-4129-84e7-a6ea39612830 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:40.257546 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7dce5f-92c5-4f02-91e8-ab2f8919f86b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:40.258178 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7dce5f-92c5-4f02-91e8-ab2f8919f86b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:40.258451 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7dce5f-92c5-4f02-91e8-ab2f8919f86b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:40.258716 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7dce5f-92c5-4f02-91e8-ab2f8919f86b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:40.261641 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4357b0e2-5d55-4129-84e7-a6ea39612830 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:40.263138 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4357b0e2-5d55-4129-84e7-a6ea39612830 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:40.268956 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7dce5f-92c5-4f02-91e8-ab2f8919f86b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:40.270397 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7dce5f-92c5-4f02-91e8-ab2f8919f86b None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:40.274575 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-4357b0e2-5d55-4129-84e7-a6ea39612830 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 06 16:03:40.274575 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:40.274575 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:40.274575 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:40.274575 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:40.274575 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:40.274575 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:40.274575 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:40.274575 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:40.274575 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:40.274575 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:40.274575 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:40.274575 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:03:40.274575 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:03:40.274575 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:03:40.274575 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:03:40.275525 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:40.275525 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:40.275525 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:03:40.275525 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:03:40.275525 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:03:40.275525 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:03:40.275525 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:40.275525 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:40.275525 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:40.275525 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:40.275525 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:40.275525 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:40.275525 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:40.275525 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:40.275525 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:40.275525 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:40.275525 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:40.275525 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:40.276472 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:40.276472 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:40.276472 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:03:40.276472 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:03:40.276472 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:03:40.276472 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:03:40.276472 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:03:40.276472 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:03:40.276472 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 06 16:03:40.276472 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:40.277133 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 360/737] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 6 16:03:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:40.278228 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4357b0e2-5d55-4129-84e7-a6ea39612830 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:40.285986 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-cf7dce5f-92c5-4f02-91e8-ab2f8919f86b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:40.285986 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:40.285986 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:40.285986 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:40.285986 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:40.285986 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:40.285986 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:40.285986 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:40.285986 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:40.285986 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:40.285986 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:40.285986 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:40.285986 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:03:40.285986 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:03:40.285986 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:03:40.285986 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:03:40.287184 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:40.287184 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:40.287184 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:40.287184 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:40.287184 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:40.287184 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:40.287184 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:40.287184 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:40.287184 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:40.287184 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:40.287184 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:40.287184 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:40.287184 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:40.287184 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:40.287184 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:40.287184 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:40.287184 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:03:40.288310 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:03:40.288310 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:40.288310 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:40.291589 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 378/738] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:03:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:40.292814 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36449dcf-74bb-4de4-919f-884e6cbfd74d None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:40.293092 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36449dcf-74bb-4de4-919f-884e6cbfd74d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:40.293355 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36449dcf-74bb-4de4-919f-884e6cbfd74d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:40.310382 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-36449dcf-74bb-4de4-919f-884e6cbfd74d None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:03:40.311546 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-36449dcf-74bb-4de4-919f-884e6cbfd74d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:03:40.344673 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:40.378978 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 379/739] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:03:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:03:40.379963 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36449dcf-74bb-4de4-919f-884e6cbfd74d None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:40.380226 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4357b0e2-5d55-4129-84e7-a6ea39612830 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:40.405095 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:40.420326 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d023962f-4e0e-4423-8120-014b134e1620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:40.421153 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d023962f-4e0e-4423-8120-014b134e1620 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:40.421569 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d023962f-4e0e-4423-8120-014b134e1620 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:40.422008 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d023962f-4e0e-4423-8120-014b134e1620 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:40.422733 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d023962f-4e0e-4423-8120-014b134e1620 None admin] RBAC: Adding query filter params (role.id=edaabdea8be84a7980fb2c651fe52f63, scope.project.id=128d210e8dad48ac921ed4cb1d444ceb, user.id=63de41f4348243998029f7d07aa5dc40) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:40.423036 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d023962f-4e0e-4423-8120-014b134e1620 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:40.424472 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d023962f-4e0e-4423-8120-014b134e1620 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:40.440781 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 361/740] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 6 16:03:40 2022] GET /identity/v3/role_assignments?user.id=63de41f4348243998029f7d07aa5dc40&scope.project.id=128d210e8dad48ac921ed4cb1d444ceb&role.id=edaabdea8be84a7980fb2c651fe52f63 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:40.440781 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d023962f-4e0e-4423-8120-014b134e1620 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:40.523185 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36449dcf-74bb-4de4-919f-884e6cbfd74d None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:40.544186 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d023962f-4e0e-4423-8120-014b134e1620 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:40.578171 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b2fc0c-c8f3-4752-a609-9431f5303ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:40.579141 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b2fc0c-c8f3-4752-a609-9431f5303ca0 None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:40.579953 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b2fc0c-c8f3-4752-a609-9431f5303ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:40.580386 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b2fc0c-c8f3-4752-a609-9431f5303ca0 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:40.581502 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b2fc0c-c8f3-4752-a609-9431f5303ca0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:40.582547 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f1fbe3-2a28-4d54-a606-48722925cd31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:40.583243 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f1fbe3-2a28-4d54-a606-48722925cd31 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:40.583588 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f1fbe3-2a28-4d54-a606-48722925cd31 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:40.583924 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f1fbe3-2a28-4d54-a606-48722925cd31 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:40.584251 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b2fc0c-c8f3-4752-a609-9431f5303ca0 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:40.584838 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f1fbe3-2a28-4d54-a606-48722925cd31 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:40.585213 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f1fbe3-2a28-4d54-a606-48722925cd31 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:40.586720 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f1fbe3-2a28-4d54-a606-48722925cd31 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:40.606182 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 380/741] 10.222.0.43 () {60 vars in 1141 bytes} [Tue Sep 6 16:03:40 2022] POST /identity/v3/services => generated 251 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:03:40.607499 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b2fc0c-c8f3-4752-a609-9431f5303ca0 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:40.608931 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 362/742] 10.222.0.43 () {56 vars in 1135 bytes} [Tue Sep 6 16:03:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:40.615330 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f1fbe3-2a28-4d54-a606-48722925cd31 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:40.651908 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:40.682689 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:40.748549 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f1fbe3-2a28-4d54-a606-48722925cd31 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:40.769181 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b2fc0c-c8f3-4752-a609-9431f5303ca0 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:40.790088 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f23a98-c46b-4b75-9ba9-5575fc7a9f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:40.790689 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f23a98-c46b-4b75-9ba9-5575fc7a9f51 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:40.790973 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f23a98-c46b-4b75-9ba9-5575fc7a9f51 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:40.791246 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f23a98-c46b-4b75-9ba9-5575fc7a9f51 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:40.812738 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c90f7cb-b17e-47e7-a210-34b3596934c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:40.813120 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c90f7cb-b17e-47e7-a210-34b3596934c8 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:40.813394 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c90f7cb-b17e-47e7-a210-34b3596934c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:40.813692 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c90f7cb-b17e-47e7-a210-34b3596934c8 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:40.814419 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c90f7cb-b17e-47e7-a210-34b3596934c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:40.814715 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c90f7cb-b17e-47e7-a210-34b3596934c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:40.816138 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c90f7cb-b17e-47e7-a210-34b3596934c8 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:40.829054 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 381/743] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:03:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:40.830252 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c90f7cb-b17e-47e7-a210-34b3596934c8 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:40.874481 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f23a98-c46b-4b75-9ba9-5575fc7a9f51 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:40.876072 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f23a98-c46b-4b75-9ba9-5575fc7a9f51 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:40.922690 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:40.929067 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:40.948419 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c90f7cb-b17e-47e7-a210-34b3596934c8 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:40.987056 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 363/744] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 6 16:03:40 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 16:03:40.988113 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f23a98-c46b-4b75-9ba9-5575fc7a9f51 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:40.997444 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31327afd-2768-47e4-b81d-295f474bb7fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:40.998073 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31327afd-2768-47e4-b81d-295f474bb7fa None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:40.998373 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31327afd-2768-47e4-b81d-295f474bb7fa None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:40.999087 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31327afd-2768-47e4-b81d-295f474bb7fa None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:40.999765 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31327afd-2768-47e4-b81d-295f474bb7fa None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:41.001925 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31327afd-2768-47e4-b81d-295f474bb7fa None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:41.023575 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 382/745] 10.222.0.43 () {60 vars in 1141 bytes} [Tue Sep 6 16:03:40 2022] POST /identity/v3/services => generated 240 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:03:41.024426 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31327afd-2768-47e4-b81d-295f474bb7fa None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:41.155783 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31327afd-2768-47e4-b81d-295f474bb7fa None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:41.166624 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f23a98-c46b-4b75-9ba9-5575fc7a9f51 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:41.202951 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41419f6-c666-4ee1-9bd2-cfc4648d447f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:41.203996 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41419f6-c666-4ee1-9bd2-cfc4648d447f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:41.204955 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41419f6-c666-4ee1-9bd2-cfc4648d447f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:41.205233 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41419f6-c666-4ee1-9bd2-cfc4648d447f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:41.214079 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927bb872-42ba-4a13-b9ea-d3807427ebef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:41.214775 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927bb872-42ba-4a13-b9ea-d3807427ebef None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:41.215068 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927bb872-42ba-4a13-b9ea-d3807427ebef None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:41.215343 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927bb872-42ba-4a13-b9ea-d3807427ebef None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:41.220320 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41419f6-c666-4ee1-9bd2-cfc4648d447f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:41.222244 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41419f6-c666-4ee1-9bd2-cfc4648d447f None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:41.224600 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927bb872-42ba-4a13-b9ea-d3807427ebef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:41.226146 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:41.226254 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927bb872-42ba-4a13-b9ea-d3807427ebef None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:41.239275 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-927bb872-42ba-4a13-b9ea-d3807427ebef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:03:41.239275 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:41.239275 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:41.239275 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:41.239275 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:41.239275 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:41.239275 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:41.239275 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:41.239275 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:41.239275 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:41.239275 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:41.239275 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:41.239275 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:03:41.239275 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:03:41.239275 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:03:41.240214 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:03:41.240214 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:41.240214 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:41.240214 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:41.240214 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:41.240214 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:41.240214 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:41.240214 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:41.240214 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:41.240214 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:41.240214 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:41.240214 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:41.240214 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:41.240214 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:41.240214 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:41.240214 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:41.240214 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:41.241135 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:03:41.241135 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:03:41.241135 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:03:41.241135 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:03:41.241135 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:03:41.241135 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:03:41.241135 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:03:41.241135 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:41.241135 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 364/746] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 6 16:03:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:41.244601 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-e41419f6-c666-4ee1-9bd2-cfc4648d447f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:41.244601 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:41.244601 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:41.244601 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:41.244601 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:41.244601 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:41.244601 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:41.244601 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:41.244601 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:41.244601 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:41.244601 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:41.244601 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:41.244601 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:03:41.244601 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:03:41.244601 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:03:41.244601 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:03:41.245648 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:41.245648 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:41.245648 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:41.245648 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:41.245648 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:41.245648 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:41.245648 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:41.245648 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:41.245648 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:41.245648 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:41.245648 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:41.245648 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:41.245648 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:41.245648 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:41.245648 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:41.245648 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:41.245648 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:03:41.246535 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:03:41.246535 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:41.246535 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:41.248311 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 383/747] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:03:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:41.251240 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927bb872-42ba-4a13-b9ea-d3807427ebef None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:41.254258 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:41.258123 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41419f6-c666-4ee1-9bd2-cfc4648d447f None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:41.364819 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927bb872-42ba-4a13-b9ea-d3807427ebef None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:41.365100 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41419f6-c666-4ee1-9bd2-cfc4648d447f None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:41.400290 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246ab68b-8014-49fc-9785-f832a0414ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:41.400931 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246ab68b-8014-49fc-9785-f832a0414ea5 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:41.401215 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246ab68b-8014-49fc-9785-f832a0414ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:41.401502 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246ab68b-8014-49fc-9785-f832a0414ea5 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:41.402113 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246ab68b-8014-49fc-9785-f832a0414ea5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:41.402216 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ca3982-3755-4a0a-8402-4d93937e5845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:41.402408 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246ab68b-8014-49fc-9785-f832a0414ea5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:41.402742 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ca3982-3755-4a0a-8402-4d93937e5845 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:41.403026 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ca3982-3755-4a0a-8402-4d93937e5845 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:41.403298 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ca3982-3755-4a0a-8402-4d93937e5845 None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:41.403770 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246ab68b-8014-49fc-9785-f832a0414ea5 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:41.403995 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ca3982-3755-4a0a-8402-4d93937e5845 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:41.404301 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ca3982-3755-4a0a-8402-4d93937e5845 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:41.405782 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ca3982-3755-4a0a-8402-4d93937e5845 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:41.416135 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 384/748] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:03:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:41.418447 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 365/749] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 6 16:03:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:41.421610 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246ab68b-8014-49fc-9785-f832a0414ea5 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:41.424066 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ca3982-3755-4a0a-8402-4d93937e5845 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:41.479350 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:41.487886 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:41.525689 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246ab68b-8014-49fc-9785-f832a0414ea5 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:41.528098 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ca3982-3755-4a0a-8402-4d93937e5845 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:41.560308 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aa15a2-bb56-43d0-9816-9a48c6912d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:41.560973 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aa15a2-bb56-43d0-9816-9a48c6912d84 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:41.561250 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aa15a2-bb56-43d0-9816-9a48c6912d84 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:41.561519 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aa15a2-bb56-43d0-9816-9a48c6912d84 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:41.564035 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e249e1-6c0a-491b-9996-549b36d87de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:41.564718 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e249e1-6c0a-491b-9996-549b36d87de4 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:41.565030 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e249e1-6c0a-491b-9996-549b36d87de4 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:41.565309 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e249e1-6c0a-491b-9996-549b36d87de4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:41.566090 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e249e1-6c0a-491b-9996-549b36d87de4 None admin] RBAC: Adding query filter params (role.id=31321627b02c4ff188f8fe1521ce3181, scope.project.id=128d210e8dad48ac921ed4cb1d444ceb, user.id=d865006413b24ec3922619311f81f353) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:41.566397 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e249e1-6c0a-491b-9996-549b36d87de4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:41.567927 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e249e1-6c0a-491b-9996-549b36d87de4 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:41.568494 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aa15a2-bb56-43d0-9816-9a48c6912d84 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:41.569960 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aa15a2-bb56-43d0-9816-9a48c6912d84 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:41.574332 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 366/750] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 6 16:03:41 2022] GET /identity/v3/role_assignments?user.id=d865006413b24ec3922619311f81f353&scope.project.id=128d210e8dad48ac921ed4cb1d444ceb&role.id=31321627b02c4ff188f8fe1521ce3181 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:03:41.582081 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-76aa15a2-bb56-43d0-9816-9a48c6912d84 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:03:41.582081 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:41.582081 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:41.582081 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:41.582081 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:41.582081 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:41.582081 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:41.582081 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:41.582081 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:41.582081 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:41.582081 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:41.582081 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:41.582081 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:03:41.582081 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:03:41.582081 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:03:41.584182 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:03:41.584182 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:41.584182 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:41.584182 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:41.584182 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:41.584182 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:41.584182 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:41.584182 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:41.584182 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:41.584182 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:41.584182 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:41.584182 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:41.584182 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:41.584182 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:41.584182 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:41.584182 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:41.584182 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:41.586357 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:03:41.586357 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:03:41.586357 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:03:41.586357 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:03:41.586357 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:03:41.586357 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:03:41.586357 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:03:41.586357 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:41.586357 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 385/751] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 6 16:03:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:41.592810 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e249e1-6c0a-491b-9996-549b36d87de4 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:41.742309 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e249e1-6c0a-491b-9996-549b36d87de4 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:41.770349 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:41.779839 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16f3d0a-62cb-4a6a-b298-fadb12d6df4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:41.780496 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16f3d0a-62cb-4a6a-b298-fadb12d6df4a None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:41.780832 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16f3d0a-62cb-4a6a-b298-fadb12d6df4a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:41.781115 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16f3d0a-62cb-4a6a-b298-fadb12d6df4a None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:41.781839 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16f3d0a-62cb-4a6a-b298-fadb12d6df4a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:41.782148 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16f3d0a-62cb-4a6a-b298-fadb12d6df4a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:41.783678 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16f3d0a-62cb-4a6a-b298-fadb12d6df4a None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:41.798547 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 367/752] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 6 16:03:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:41.804187 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aa15a2-bb56-43d0-9816-9a48c6912d84 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:41.958290 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aa15a2-bb56-43d0-9816-9a48c6912d84 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:41.975353 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:42.004042 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f973421-8038-4800-93c1-c779012180db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:42.004568 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f973421-8038-4800-93c1-c779012180db None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:42.004868 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f973421-8038-4800-93c1-c779012180db None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:42.005136 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f973421-8038-4800-93c1-c779012180db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:42.005856 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f973421-8038-4800-93c1-c779012180db None admin] RBAC: Adding query filter params (role.id=edaabdea8be84a7980fb2c651fe52f63, scope.project.id=128d210e8dad48ac921ed4cb1d444ceb, user.id=312026c080b64aff9dd483d3591288a6) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:42.006157 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f973421-8038-4800-93c1-c779012180db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:42.007586 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f973421-8038-4800-93c1-c779012180db None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:42.014988 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 386/753] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 6 16:03:41 2022] GET /identity/v3/role_assignments?user.id=312026c080b64aff9dd483d3591288a6&scope.project.id=128d210e8dad48ac921ed4cb1d444ceb&role.id=edaabdea8be84a7980fb2c651fe52f63 => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:42.562953 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a57ef8d-55c7-427c-99ec-73248977589a None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:42.563154 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a57ef8d-55c7-427c-99ec-73248977589a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:42.563435 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a57ef8d-55c7-427c-99ec-73248977589a None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:42.565232 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 368/754] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:03:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:03:42.570937 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc24510-4168-45a0-aaca-0c1aac57e0da None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:42.571430 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc24510-4168-45a0-aaca-0c1aac57e0da None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:42.571506 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc24510-4168-45a0-aaca-0c1aac57e0da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:42.598256 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-7bc24510-4168-45a0-aaca-0c1aac57e0da None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:03:42.599466 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7bc24510-4168-45a0-aaca-0c1aac57e0da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:03:42.695103 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 387/755] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:03:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:03:42.848549 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f2dc04-1474-4208-8842-050fd95773d1 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:42.849080 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f2dc04-1474-4208-8842-050fd95773d1 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:42.849080 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f2dc04-1474-4208-8842-050fd95773d1 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:42.850667 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 369/756] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:03:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:03:42.856001 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893d1f83-c4b5-4c63-84e2-95fe993f941e None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:42.856317 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893d1f83-c4b5-4c63-84e2-95fe993f941e None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:42.856561 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893d1f83-c4b5-4c63-84e2-95fe993f941e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:42.885787 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-893d1f83-c4b5-4c63-84e2-95fe993f941e None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:03:42.886986 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-893d1f83-c4b5-4c63-84e2-95fe993f941e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:03:42.980348 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 388/757] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:03:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:03:43.028494 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f2dc04-1474-4208-8842-050fd95773d1 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:43.185409 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f2dc04-1474-4208-8842-050fd95773d1 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:43.234129 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e95e2c-da9c-4e82-b450-7e90fc716742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:43.234768 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e95e2c-da9c-4e82-b450-7e90fc716742 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:43.235046 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e95e2c-da9c-4e82-b450-7e90fc716742 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:43.235315 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e95e2c-da9c-4e82-b450-7e90fc716742 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:43.263693 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e95e2c-da9c-4e82-b450-7e90fc716742 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:03:43.268670 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-53e95e2c-da9c-4e82-b450-7e90fc716742 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:03:43.268670 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:43.268670 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:43.268670 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:43.268670 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:43.268670 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:43.268670 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:43.268670 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:43.268670 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:43.268670 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:43.268670 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:43.268670 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:43.268670 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:03:43.268670 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:03:43.268670 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:03:43.268670 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:03:43.269700 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:03:43.269700 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:03:43.269700 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:03:43.269700 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:03:43.269700 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:43.269700 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:43.269700 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:43.269700 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:43.269700 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:43.269700 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:43.269700 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:43.269700 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:43.269700 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:43.269700 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:43.269700 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:43.269700 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:43.270556 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:43.270556 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:43.270556 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:43.270556 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:43.270556 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:03:43.270556 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:03:43.270556 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:03:43.270556 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:03:43.270556 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:03:43.270556 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:03:43.270556 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:03:43.270556 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:43.270556 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 370/758] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 6 16:03:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:43.280817 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893d1f83-c4b5-4c63-84e2-95fe993f941e None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:43.347083 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:43.381228 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e95e2c-da9c-4e82-b450-7e90fc716742 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:43.435608 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:43.451660 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893d1f83-c4b5-4c63-84e2-95fe993f941e None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:43.494063 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e95e2c-da9c-4e82-b450-7e90fc716742 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:43.532685 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928bd869-1762-48b8-82c2-2a26ef98953f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:43.533696 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928bd869-1762-48b8-82c2-2a26ef98953f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:43.534263 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928bd869-1762-48b8-82c2-2a26ef98953f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:43.534740 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928bd869-1762-48b8-82c2-2a26ef98953f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:43.535558 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928bd869-1762-48b8-82c2-2a26ef98953f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:43.537294 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928bd869-1762-48b8-82c2-2a26ef98953f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:43.547477 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-928bd869-1762-48b8-82c2-2a26ef98953f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 16:03:43.547477 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:43.547477 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 16:03:43.547477 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 16:03:43.547477 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 16:03:43.547477 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 16:03:43.547477 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 16:03:43.547477 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:03:43.547477 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 16:03:43.547477 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:43.547477 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 16:03:43.547477 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:43.547477 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:43.547477 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:43.547477 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:43.547477 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:43.547477 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:43.548534 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:43.548534 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:43.548534 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:43.548534 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:43.548534 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:43.548534 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:43.548534 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 16:03:43.548534 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 16:03:43.548534 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 16:03:43.548534 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 16:03:43.548534 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:43.548534 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:43.548534 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:43.548534 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:43.548534 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:43.548534 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:43.549408 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:43.549408 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:43.549408 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:43.549408 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:43.549408 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:43.549408 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:43.549408 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:43.549408 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:43.549408 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:43.549408 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:43.549408 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 16:03:43.549408 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:03:43.549408 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 16:03:43.549408 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:43.552233 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 371/759] 10.222.0.43 () {56 vars in 1129 bytes} [Tue Sep 6 16:03:43 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:03:43.558387 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928bd869-1762-48b8-82c2-2a26ef98953f None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:43.577502 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a43864d-fac1-40c0-8033-3731f98df77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:43.578125 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a43864d-fac1-40c0-8033-3731f98df77b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:43.578561 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a43864d-fac1-40c0-8033-3731f98df77b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:43.578968 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a43864d-fac1-40c0-8033-3731f98df77b None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:43.580089 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a43864d-fac1-40c0-8033-3731f98df77b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:43.580553 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a43864d-fac1-40c0-8033-3731f98df77b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:43.582046 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a43864d-fac1-40c0-8033-3731f98df77b None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:43.607048 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 389/760] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 6 16:03:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:43.617817 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a43864d-fac1-40c0-8033-3731f98df77b None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:43.682166 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928bd869-1762-48b8-82c2-2a26ef98953f None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:43.697204 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:43.733289 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cf193c-4128-4914-a823-161741026115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:43.734576 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cf193c-4128-4914-a823-161741026115 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:43.735091 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cf193c-4128-4914-a823-161741026115 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:43.736034 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cf193c-4128-4914-a823-161741026115 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:43.737204 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cf193c-4128-4914-a823-161741026115 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:43.738050 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cf193c-4128-4914-a823-161741026115 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:43.740131 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cf193c-4128-4914-a823-161741026115 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:43.752199 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 372/761] 10.222.0.43 () {56 vars in 1124 bytes} [Tue Sep 6 16:03:43 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:03:43.752199 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964aff14-5120-4637-aadf-6e9559c89e2b None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:43.752199 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964aff14-5120-4637-aadf-6e9559c89e2b None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:43.752199 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964aff14-5120-4637-aadf-6e9559c89e2b None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:43.756464 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 373/762] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:03:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:03:43.757788 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964aff14-5120-4637-aadf-6e9559c89e2b None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:43.759848 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:43.763230 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a43864d-fac1-40c0-8033-3731f98df77b None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:43.802518 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7048ba60-8809-48ab-af39-9b73162bf1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:43.803784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7048ba60-8809-48ab-af39-9b73162bf1a0 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:43.804520 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7048ba60-8809-48ab-af39-9b73162bf1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:43.804960 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7048ba60-8809-48ab-af39-9b73162bf1a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:43.818255 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7048ba60-8809-48ab-af39-9b73162bf1a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:43.820607 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7048ba60-8809-48ab-af39-9b73162bf1a0 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:43.839801 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-7048ba60-8809-48ab-af39-9b73162bf1a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:43.839801 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:43.839801 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:43.839801 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:43.839801 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:43.839801 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:43.839801 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:43.839801 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:43.839801 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:43.839801 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:43.839801 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:43.839801 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:43.839801 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:03:43.839801 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:03:43.839801 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:03:43.839801 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:03:43.840942 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:43.840942 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:43.840942 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:43.840942 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:43.840942 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:43.840942 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:43.840942 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:43.840942 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:43.840942 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:43.840942 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:43.840942 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:43.840942 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:43.840942 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:43.840942 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:43.840942 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:43.840942 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:43.840942 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:03:43.841847 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:03:43.841847 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:43.841847 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:43.842015 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 390/763] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:03:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:43.843156 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce4806-e7f4-4be0-bf23-57cc3c390d3f None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:43.843438 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce4806-e7f4-4be0-bf23-57cc3c390d3f None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:43.843708 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce4806-e7f4-4be0-bf23-57cc3c390d3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:43.865662 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-3bce4806-e7f4-4be0-bf23-57cc3c390d3f None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:03:43.867186 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-3bce4806-e7f4-4be0-bf23-57cc3c390d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:03:43.897604 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964aff14-5120-4637-aadf-6e9559c89e2b None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:43.947345 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf98f93-c539-45d5-851a-4d958f8da213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:43.948025 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf98f93-c539-45d5-851a-4d958f8da213 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:43.948308 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf98f93-c539-45d5-851a-4d958f8da213 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:43.948583 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf98f93-c539-45d5-851a-4d958f8da213 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:43.949400 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf98f93-c539-45d5-851a-4d958f8da213 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:43.949705 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf98f93-c539-45d5-851a-4d958f8da213 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:43.950900 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 391/764] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:03:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:03:43.951095 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf98f93-c539-45d5-851a-4d958f8da213 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:43.951915 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bce4806-e7f4-4be0-bf23-57cc3c390d3f None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:43.958439 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 374/765] 10.222.0.43 () {56 vars in 1124 bytes} [Tue Sep 6 16:03:43 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:43.964201 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf98f93-c539-45d5-851a-4d958f8da213 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:44.005673 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:44.035956 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:44.055466 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bce4806-e7f4-4be0-bf23-57cc3c390d3f None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:44.071447 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf98f93-c539-45d5-851a-4d958f8da213 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:44.091340 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499c987d-64a8-4e61-9b6a-a546873a4385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:44.092104 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499c987d-64a8-4e61-9b6a-a546873a4385 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:44.092465 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499c987d-64a8-4e61-9b6a-a546873a4385 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:44.092885 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499c987d-64a8-4e61-9b6a-a546873a4385 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:44.093623 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499c987d-64a8-4e61-9b6a-a546873a4385 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:44.094014 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499c987d-64a8-4e61-9b6a-a546873a4385 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:44.095483 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499c987d-64a8-4e61-9b6a-a546873a4385 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:44.112804 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 392/766] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:03:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:44.112804 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b1b23f-6d67-4c84-b0ca-a6ff202e8d72 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:44.112804 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b1b23f-6d67-4c84-b0ca-a6ff202e8d72 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:44.112804 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b1b23f-6d67-4c84-b0ca-a6ff202e8d72 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:44.112804 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 393/767] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:03:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:03:44.114172 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa98583-99d5-4d1a-aa8a-a60f20d6e0e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:44.114783 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa98583-99d5-4d1a-aa8a-a60f20d6e0e9 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:44.115062 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa98583-99d5-4d1a-aa8a-a60f20d6e0e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:44.115329 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa98583-99d5-4d1a-aa8a-a60f20d6e0e9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:44.116034 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa98583-99d5-4d1a-aa8a-a60f20d6e0e9 None admin] RBAC: Adding query filter params (interface=public, service_id=5077b95cb83c4f73a6fd4736902fe585, region_id=RegionOne) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:44.116331 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa98583-99d5-4d1a-aa8a-a60f20d6e0e9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:44.117740 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa98583-99d5-4d1a-aa8a-a60f20d6e0e9 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:44.120218 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b1b23f-6d67-4c84-b0ca-a6ff202e8d72 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:44.128328 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 375/768] 10.222.0.43 () {56 vars in 1251 bytes} [Tue Sep 6 16:03:43 2022] GET /identity/v3/endpoints?service_id=5077b95cb83c4f73a6fd4736902fe585&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:44.133005 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35105bd7-69e0-4a9c-bc18-46b1157ec1ea None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:44.133005 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35105bd7-69e0-4a9c-bc18-46b1157ec1ea None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:44.133005 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35105bd7-69e0-4a9c-bc18-46b1157ec1ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:44.153173 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-35105bd7-69e0-4a9c-bc18-46b1157ec1ea None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:03:44.154846 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-35105bd7-69e0-4a9c-bc18-46b1157ec1ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:03:44.229134 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b1b23f-6d67-4c84-b0ca-a6ff202e8d72 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:44.231479 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 376/769] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:03:44 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:03:44.232653 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35105bd7-69e0-4a9c-bc18-46b1157ec1ea None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:44.271470 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2173c16-91b9-4cbe-a67b-828937d0ec7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:44.271993 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2173c16-91b9-4cbe-a67b-828937d0ec7b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:44.272264 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2173c16-91b9-4cbe-a67b-828937d0ec7b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:44.272529 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2173c16-91b9-4cbe-a67b-828937d0ec7b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:44.283685 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:44.287417 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2173c16-91b9-4cbe-a67b-828937d0ec7b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:44.290116 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2173c16-91b9-4cbe-a67b-828937d0ec7b None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:44.311496 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-d2173c16-91b9-4cbe-a67b-828937d0ec7b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 16:03:44.311496 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:44.311496 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:44.311496 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:44.311496 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:44.311496 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:44.311496 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:44.311496 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:44.311496 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:44.311496 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:44.311496 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:44.311496 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:44.311496 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:03:44.311496 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:03:44.311496 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:03:44.311496 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:03:44.312501 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:44.312501 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:44.312501 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:03:44.312501 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:03:44.312501 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:03:44.312501 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:03:44.312501 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:44.312501 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:44.312501 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:44.312501 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:44.312501 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:44.312501 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:44.312501 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:44.312501 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:44.312501 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:44.312501 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:44.312501 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:44.312501 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:44.313497 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:44.313497 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:44.313497 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:03:44.313497 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:03:44.313497 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:03:44.313497 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:03:44.313497 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:03:44.313497 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:03:44.313497 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 16:03:44.313497 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:44.314645 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 394/770] 10.222.0.43 () {56 vars in 1093 bytes} [Tue Sep 6 16:03:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:44.319600 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2173c16-91b9-4cbe-a67b-828937d0ec7b None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:44.321463 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:44.373165 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35105bd7-69e0-4a9c-bc18-46b1157ec1ea None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:44.410392 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c1bab6-7eb2-41d9-ada8-82fd83f8fe52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:44.411283 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c1bab6-7eb2-41d9-ada8-82fd83f8fe52 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:44.411738 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c1bab6-7eb2-41d9-ada8-82fd83f8fe52 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:44.412181 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c1bab6-7eb2-41d9-ada8-82fd83f8fe52 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:44.413029 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c1bab6-7eb2-41d9-ada8-82fd83f8fe52 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:44.415059 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c1bab6-7eb2-41d9-ada8-82fd83f8fe52 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:44.428860 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 377/771] 10.222.0.43 () {56 vars in 1087 bytes} [Tue Sep 6 16:03:44 2022] GET /identity/v3/services => generated 772 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:44.430983 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c1bab6-7eb2-41d9-ada8-82fd83f8fe52 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:44.481316 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2173c16-91b9-4cbe-a67b-828937d0ec7b None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:44.559506 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061972ac-8637-4b0d-986c-712306de1184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:44.560171 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061972ac-8637-4b0d-986c-712306de1184 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:44.560446 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061972ac-8637-4b0d-986c-712306de1184 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:44.560716 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061972ac-8637-4b0d-986c-712306de1184 None admin] PATH_INFO: `/v3/services/network` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:44.561414 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061972ac-8637-4b0d-986c-712306de1184 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:44.562912 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061972ac-8637-4b0d-986c-712306de1184 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:44.579710 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-061972ac-8637-4b0d-986c-712306de1184 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 16:03:44.579710 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:44.579710 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 16:03:44.579710 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 16:03:44.579710 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 16:03:44.579710 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 16:03:44.579710 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 16:03:44.579710 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:03:44.579710 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 16:03:44.579710 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:44.579710 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 16:03:44.579710 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:44.579710 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:44.579710 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:44.579710 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:44.579710 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:44.579710 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:44.581400 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:44.581400 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:44.581400 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:44.581400 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:44.581400 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:44.581400 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:44.581400 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 16:03:44.581400 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 16:03:44.581400 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 16:03:44.581400 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 16:03:44.581400 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:44.581400 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:44.581400 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:44.581400 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:44.581400 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:44.581400 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:44.582233 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:44.582233 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:44.582233 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:44.582233 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:44.582233 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:44.582233 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:44.582233 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:44.582233 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:44.582233 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:44.582233 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:44.582233 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 16:03:44.582233 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:03:44.582233 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 16:03:44.582233 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:44.582233 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 395/772] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 6 16:03:44 2022] GET /identity/v3/services/network => generated 88 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:44.584064 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061972ac-8637-4b0d-986c-712306de1184 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:44.585953 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:44.592729 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c1bab6-7eb2-41d9-ada8-82fd83f8fe52 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:44.629041 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2a23c1-fe83-4bcb-b22c-a7d3436ae120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:44.629738 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a23c1-fe83-4bcb-b22c-a7d3436ae120 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:44.630011 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a23c1-fe83-4bcb-b22c-a7d3436ae120 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:44.630280 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a23c1-fe83-4bcb-b22c-a7d3436ae120 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:44.631025 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2a23c1-fe83-4bcb-b22c-a7d3436ae120 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:44.631317 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2a23c1-fe83-4bcb-b22c-a7d3436ae120 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:44.632836 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2a23c1-fe83-4bcb-b22c-a7d3436ae120 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:44.650172 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 378/773] 10.222.0.43 () {56 vars in 1133 bytes} [Tue Sep 6 16:03:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:44.651440 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b53fd1a-817d-48f9-b15f-436d18fc02df None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:44.651729 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b53fd1a-817d-48f9-b15f-436d18fc02df None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:44.652001 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b53fd1a-817d-48f9-b15f-436d18fc02df None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:44.656918 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 379/774] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:03:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:03:44.656918 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b53fd1a-817d-48f9-b15f-436d18fc02df None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:44.666127 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:44.705173 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061972ac-8637-4b0d-986c-712306de1184 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:44.741324 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091cd01a-ffd0-4fb6-8009-6caced27d9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:44.741861 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091cd01a-ffd0-4fb6-8009-6caced27d9ac None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:44.742139 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091cd01a-ffd0-4fb6-8009-6caced27d9ac None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:44.742410 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091cd01a-ffd0-4fb6-8009-6caced27d9ac None admin] PATH_INFO: `/v3/roles/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:44.753810 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091cd01a-ffd0-4fb6-8009-6caced27d9ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:03:44.758699 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-091cd01a-ffd0-4fb6-8009-6caced27d9ac None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:03:44.758699 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:44.758699 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:44.758699 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:44.758699 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:44.758699 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:44.758699 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:44.758699 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:44.758699 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:44.758699 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:44.758699 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:44.758699 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:44.758699 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:03:44.758699 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:03:44.758699 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:03:44.758699 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:03:44.759626 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:03:44.759626 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:03:44.759626 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:03:44.759626 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:03:44.759626 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:44.759626 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:44.759626 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:44.759626 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:44.759626 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:44.759626 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:44.759626 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:44.759626 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:44.759626 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:44.759626 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:44.759626 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:44.759626 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:44.760450 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:44.760450 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:44.760450 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:44.760450 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:44.760450 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:03:44.760450 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:03:44.760450 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:03:44.760450 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:03:44.760450 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:03:44.760450 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:03:44.760450 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:03:44.760450 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:44.760450 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 396/775] 10.222.0.43 () {56 vars in 1102 bytes} [Tue Sep 6 16:03:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:44.761181 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091cd01a-ffd0-4fb6-8009-6caced27d9ac None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:44.767656 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b53fd1a-817d-48f9-b15f-436d18fc02df None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:44.802032 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac83206-1f61-4c41-853e-389b943e9c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:44.802651 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac83206-1f61-4c41-853e-389b943e9c89 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:44.802927 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac83206-1f61-4c41-853e-389b943e9c89 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:44.803198 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac83206-1f61-4c41-853e-389b943e9c89 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:44.803847 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac83206-1f61-4c41-853e-389b943e9c89 None admin] RBAC: Adding query filter params (name=network) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:44.804146 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac83206-1f61-4c41-853e-389b943e9c89 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:44.805560 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac83206-1f61-4c41-853e-389b943e9c89 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:44.810097 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 380/776] 10.222.0.43 () {56 vars in 1112 bytes} [Tue Sep 6 16:03:44 2022] GET /identity/v3/services?name=network => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:44.811295 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b9fac7-90e3-4bb2-8682-597ffe1b7fea None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:44.811576 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b9fac7-90e3-4bb2-8682-597ffe1b7fea None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:44.811845 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b9fac7-90e3-4bb2-8682-597ffe1b7fea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:44.834543 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-b9b9fac7-90e3-4bb2-8682-597ffe1b7fea None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:03:44.835746 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b9b9fac7-90e3-4bb2-8682-597ffe1b7fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:03:44.849251 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:44.871248 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091cd01a-ffd0-4fb6-8009-6caced27d9ac None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:44.900169 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:44.907093 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 381/777] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:03:44 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:03:44.908027 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b9fac7-90e3-4bb2-8682-597ffe1b7fea None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:44.918513 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18df8bd-ba9c-4c85-aa9a-dfc06093116b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:44.919451 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18df8bd-ba9c-4c85-aa9a-dfc06093116b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:44.920056 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18df8bd-ba9c-4c85-aa9a-dfc06093116b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:44.920529 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18df8bd-ba9c-4c85-aa9a-dfc06093116b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:44.932422 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18df8bd-ba9c-4c85-aa9a-dfc06093116b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:44.934316 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18df8bd-ba9c-4c85-aa9a-dfc06093116b None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:44.950448 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-a18df8bd-ba9c-4c85-aa9a-dfc06093116b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:44.950448 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:44.950448 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:44.950448 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:44.950448 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:44.950448 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:44.950448 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:44.950448 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:44.950448 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:44.950448 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:44.950448 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:44.950448 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:44.950448 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:03:44.950448 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:03:44.950448 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:03:44.950448 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:03:44.951443 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:44.951443 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:44.951443 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:44.951443 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:44.951443 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:44.951443 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:44.951443 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:44.951443 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:44.951443 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:44.951443 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:44.951443 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:44.951443 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:44.951443 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:44.951443 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:44.951443 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:44.951443 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:44.951443 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:03:44.952340 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:03:44.952340 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:44.952340 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:44.959688 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 397/778] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:03:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:44.960917 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18df8bd-ba9c-4c85-aa9a-dfc06093116b None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:45.009232 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b9fac7-90e3-4bb2-8682-597ffe1b7fea None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:45.043729 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c88fed-b7c1-4f0e-9755-529368eff873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:45.044344 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c88fed-b7c1-4f0e-9755-529368eff873 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:45.044618 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c88fed-b7c1-4f0e-9755-529368eff873 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:45.044973 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c88fed-b7c1-4f0e-9755-529368eff873 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:45.045645 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c88fed-b7c1-4f0e-9755-529368eff873 None admin] RBAC: Adding query filter params (name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:45.045954 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c88fed-b7c1-4f0e-9755-529368eff873 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:45.047326 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c88fed-b7c1-4f0e-9755-529368eff873 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:45.060585 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 382/779] 10.222.0.43 () {56 vars in 1103 bytes} [Tue Sep 6 16:03:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:45.061522 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c88fed-b7c1-4f0e-9755-529368eff873 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:45.069338 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18df8bd-ba9c-4c85-aa9a-dfc06093116b None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:45.074386 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:45.108000 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da491cd-6839-4622-96ed-d713d63cc6c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:45.109127 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da491cd-6839-4622-96ed-d713d63cc6c5 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:45.109712 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da491cd-6839-4622-96ed-d713d63cc6c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:45.110230 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da491cd-6839-4622-96ed-d713d63cc6c5 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:45.111054 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da491cd-6839-4622-96ed-d713d63cc6c5 None admin] RBAC: Adding query filter params (type=network) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:45.111550 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da491cd-6839-4622-96ed-d713d63cc6c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:45.113456 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da491cd-6839-4622-96ed-d713d63cc6c5 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:45.120705 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 398/780] 10.222.0.43 () {56 vars in 1112 bytes} [Tue Sep 6 16:03:44 2022] GET /identity/v3/services?type=network => generated 350 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:45.122067 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da491cd-6839-4622-96ed-d713d63cc6c5 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:45.163843 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c88fed-b7c1-4f0e-9755-529368eff873 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:45.167393 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:45.197720 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69930d35-f76d-4126-aa75-b4d19f990673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:45.198334 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69930d35-f76d-4126-aa75-b4d19f990673 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:45.198654 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69930d35-f76d-4126-aa75-b4d19f990673 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:45.198931 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69930d35-f76d-4126-aa75-b4d19f990673 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:45.199583 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69930d35-f76d-4126-aa75-b4d19f990673 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:45.199881 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69930d35-f76d-4126-aa75-b4d19f990673 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:45.201298 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69930d35-f76d-4126-aa75-b4d19f990673 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:45.214050 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 383/781] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:03:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:45.214867 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69930d35-f76d-4126-aa75-b4d19f990673 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:45.241492 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da491cd-6839-4622-96ed-d713d63cc6c5 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:45.286805 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520701e2-a7ce-43ae-a896-368e1422b77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:45.287454 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520701e2-a7ce-43ae-a896-368e1422b77b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:45.287732 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520701e2-a7ce-43ae-a896-368e1422b77b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:45.287998 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520701e2-a7ce-43ae-a896-368e1422b77b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:45.288706 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:45.300260 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520701e2-a7ce-43ae-a896-368e1422b77b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:45.301754 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520701e2-a7ce-43ae-a896-368e1422b77b None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:45.316972 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-520701e2-a7ce-43ae-a896-368e1422b77b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:45.316972 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:45.316972 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:45.316972 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:45.316972 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:45.316972 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:45.316972 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:45.316972 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:45.316972 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:45.316972 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:45.316972 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:45.316972 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:45.316972 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:03:45.316972 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:03:45.316972 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:03:45.316972 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:03:45.318170 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:45.318170 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:45.318170 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:45.318170 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:45.318170 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:45.318170 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:45.318170 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:45.318170 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:45.318170 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:45.318170 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:45.318170 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:45.318170 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:45.318170 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:45.318170 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:45.318170 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:45.318170 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:45.318170 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:03:45.319148 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:03:45.319148 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:45.319148 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:45.319148 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 399/782] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:03:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:45.319148 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520701e2-a7ce-43ae-a896-368e1422b77b None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:45.328219 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69930d35-f76d-4126-aa75-b4d19f990673 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:45.363090 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f636fae8-d3b0-4435-b12f-df1311a13640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:45.363698 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f636fae8-d3b0-4435-b12f-df1311a13640 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:45.363978 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f636fae8-d3b0-4435-b12f-df1311a13640 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:45.364246 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f636fae8-d3b0-4435-b12f-df1311a13640 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:45.364980 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f636fae8-d3b0-4435-b12f-df1311a13640 None admin] RBAC: Adding query filter params (interface=public, service_id=7df3ef040cc049499f40e12af578096c, region_id=RegionOne) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:45.365288 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f636fae8-d3b0-4435-b12f-df1311a13640 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:45.367182 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f636fae8-d3b0-4435-b12f-df1311a13640 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:45.376622 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 384/783] 10.222.0.43 () {56 vars in 1251 bytes} [Tue Sep 6 16:03:45 2022] GET /identity/v3/endpoints?service_id=7df3ef040cc049499f40e12af578096c&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:45.377504 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f636fae8-d3b0-4435-b12f-df1311a13640 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:45.421568 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:45.429349 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520701e2-a7ce-43ae-a896-368e1422b77b None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:45.467518 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca543dc-dbd1-42db-b57f-84fc00b3c234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:45.468163 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca543dc-dbd1-42db-b57f-84fc00b3c234 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:45.468440 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca543dc-dbd1-42db-b57f-84fc00b3c234 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:45.468707 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca543dc-dbd1-42db-b57f-84fc00b3c234 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:45.476617 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca543dc-dbd1-42db-b57f-84fc00b3c234 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:45.478124 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca543dc-dbd1-42db-b57f-84fc00b3c234 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:45.482938 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f636fae8-d3b0-4435-b12f-df1311a13640 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:45.490378 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-eca543dc-dbd1-42db-b57f-84fc00b3c234 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:03:45.490378 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:45.490378 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:45.490378 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:45.490378 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:45.490378 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:45.490378 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:45.490378 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:45.490378 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:45.490378 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:45.490378 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:45.490378 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:45.490378 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:03:45.490378 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:03:45.490378 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:03:45.491299 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:03:45.491299 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:45.491299 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:45.491299 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:45.491299 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:45.491299 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:45.491299 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:45.491299 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:45.491299 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:45.491299 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:45.491299 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:45.491299 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:45.491299 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:45.491299 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:45.491299 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:45.491299 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:45.491299 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:45.492599 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:03:45.492599 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:03:45.492599 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:03:45.492599 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:03:45.492599 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:03:45.492599 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:03:45.492599 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:03:45.492599 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:45.492599 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 400/784] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 6 16:03:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:45.492599 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca543dc-dbd1-42db-b57f-84fc00b3c234 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:45.523314 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a25b59-3537-45a1-b423-3e10ac6b8f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:45.523977 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a25b59-3537-45a1-b423-3e10ac6b8f61 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:45.524256 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a25b59-3537-45a1-b423-3e10ac6b8f61 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:45.524529 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a25b59-3537-45a1-b423-3e10ac6b8f61 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:45.527721 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:45.537270 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a25b59-3537-45a1-b423-3e10ac6b8f61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:03:45.542772 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-a1a25b59-3537-45a1-b423-3e10ac6b8f61 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:03:45.542772 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:45.542772 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:45.542772 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:45.542772 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:45.542772 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:45.542772 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:45.542772 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:45.542772 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:45.542772 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:45.542772 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:45.542772 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:45.542772 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:03:45.542772 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:03:45.542772 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:03:45.542772 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:03:45.543690 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:03:45.543690 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:03:45.543690 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:03:45.543690 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:03:45.543690 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:45.543690 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:45.543690 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:45.543690 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:45.543690 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:45.543690 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:45.543690 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:45.543690 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:45.543690 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:45.543690 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:45.543690 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:45.543690 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:45.544518 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:45.544518 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:45.544518 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:45.544518 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:45.544518 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:03:45.544518 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:03:45.544518 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:03:45.544518 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:03:45.544518 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:03:45.544518 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:03:45.544518 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:03:45.544518 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:45.545884 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 385/785] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 6 16:03:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:45.545884 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a25b59-3537-45a1-b423-3e10ac6b8f61 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:45.610811 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca543dc-dbd1-42db-b57f-84fc00b3c234 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:45.646216 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a25b59-3537-45a1-b423-3e10ac6b8f61 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:45.647257 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8270b64d-e05a-4518-9196-212bb33be103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:45.648019 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8270b64d-e05a-4518-9196-212bb33be103 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:45.648296 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8270b64d-e05a-4518-9196-212bb33be103 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:45.649319 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8270b64d-e05a-4518-9196-212bb33be103 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:45.650071 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8270b64d-e05a-4518-9196-212bb33be103 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:45.650375 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8270b64d-e05a-4518-9196-212bb33be103 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:45.651740 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8270b64d-e05a-4518-9196-212bb33be103 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:45.666208 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 401/786] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:03:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:45.667076 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8270b64d-e05a-4518-9196-212bb33be103 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:45.682205 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:45.686719 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9210ce7e-e573-4c31-9529-135a4e637cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:45.687335 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9210ce7e-e573-4c31-9529-135a4e637cdf None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:45.687619 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9210ce7e-e573-4c31-9529-135a4e637cdf None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:45.687883 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9210ce7e-e573-4c31-9529-135a4e637cdf None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:45.688926 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9210ce7e-e573-4c31-9529-135a4e637cdf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:45.690601 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9210ce7e-e573-4c31-9529-135a4e637cdf None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:45.700919 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 386/787] 10.222.0.43 () {56 vars in 1087 bytes} [Tue Sep 6 16:03:45 2022] GET /identity/v3/services => generated 772 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:45.701801 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9210ce7e-e573-4c31-9529-135a4e637cdf None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:45.772328 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:45.801537 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8270b64d-e05a-4518-9196-212bb33be103 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:45.819237 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9210ce7e-e573-4c31-9529-135a4e637cdf None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:45.848138 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9221ba-b37f-4ac2-94b3-f1661a64ecb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:45.848604 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9221ba-b37f-4ac2-94b3-f1661a64ecb2 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:45.848943 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9221ba-b37f-4ac2-94b3-f1661a64ecb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:45.849203 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9221ba-b37f-4ac2-94b3-f1661a64ecb2 None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:45.849964 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9221ba-b37f-4ac2-94b3-f1661a64ecb2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:45.850315 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9221ba-b37f-4ac2-94b3-f1661a64ecb2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:45.851813 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9221ba-b37f-4ac2-94b3-f1661a64ecb2 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:45.872521 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 402/788] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 6 16:03:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:45.872521 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9221ba-b37f-4ac2-94b3-f1661a64ecb2 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:45.872822 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ec0fd2-ca54-4833-b1c0-5171193138e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:45.872822 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ec0fd2-ca54-4833-b1c0-5171193138e7 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:45.872822 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ec0fd2-ca54-4833-b1c0-5171193138e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:45.872822 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ec0fd2-ca54-4833-b1c0-5171193138e7 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:45.873176 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ec0fd2-ca54-4833-b1c0-5171193138e7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:45.873176 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ec0fd2-ca54-4833-b1c0-5171193138e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:45.873514 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ec0fd2-ca54-4833-b1c0-5171193138e7 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:45.885693 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 387/789] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 6 16:03:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:45.886530 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ec0fd2-ca54-4833-b1c0-5171193138e7 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:45.963749 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:45.992647 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9221ba-b37f-4ac2-94b3-f1661a64ecb2 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:46.006144 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ec0fd2-ca54-4833-b1c0-5171193138e7 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:46.027419 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b22fdb-2020-4294-8260-8a4504e452e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:46.028064 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b22fdb-2020-4294-8260-8a4504e452e3 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:46.028337 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b22fdb-2020-4294-8260-8a4504e452e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:46.028560 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:46.028664 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b22fdb-2020-4294-8260-8a4504e452e3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:46.037862 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b22fdb-2020-4294-8260-8a4504e452e3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:46.039323 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b22fdb-2020-4294-8260-8a4504e452e3 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:46.044506 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77da122-e4b5-4c90-a2ee-e038bcb28965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:46.045272 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77da122-e4b5-4c90-a2ee-e038bcb28965 None admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:46.045558 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77da122-e4b5-4c90-a2ee-e038bcb28965 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:46.045935 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77da122-e4b5-4c90-a2ee-e038bcb28965 None admin] PATH_INFO: `/v3/projects/128d210e8dad48ac921ed4cb1d444ceb/users/63de41f4348243998029f7d07aa5dc40/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:46.047873 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77da122-e4b5-4c90-a2ee-e038bcb28965 None admin] RBAC: Authorizing `identity:create_grant(project_id=128d210e8dad48ac921ed4cb1d444ceb, user_id=63de41f4348243998029f7d07aa5dc40, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:46.049356 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77da122-e4b5-4c90-a2ee-e038bcb28965 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:46.050887 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-c7b22fdb-2020-4294-8260-8a4504e452e3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 16:03:46.050887 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:46.050887 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:46.050887 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:46.050887 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:46.050887 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:46.050887 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:46.050887 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:46.050887 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:46.050887 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:46.050887 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:46.050887 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:46.050887 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:03:46.050887 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:03:46.050887 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:03:46.050887 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:03:46.051863 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:46.051863 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:46.051863 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:03:46.051863 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:03:46.051863 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:03:46.051863 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:03:46.051863 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:46.051863 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:46.051863 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:46.051863 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:46.051863 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:46.051863 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:46.051863 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:46.051863 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:46.051863 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:46.051863 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:46.051863 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:46.051863 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:46.053622 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:46.053622 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:46.053622 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:03:46.053622 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:03:46.053622 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:03:46.053622 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:03:46.053622 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:03:46.053622 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:03:46.053622 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 16:03:46.053622 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:46.053622 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 403/790] 10.222.0.43 () {56 vars in 1117 bytes} [Tue Sep 6 16:03:45 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:46.053622 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b22fdb-2020-4294-8260-8a4504e452e3 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:46.069088 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 388/791] 10.222.0.43 () {58 vars in 1439 bytes} [Tue Sep 6 16:03:45 2022] PUT /identity/v3/projects/128d210e8dad48ac921ed4cb1d444ceb/users/63de41f4348243998029f7d07aa5dc40/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:03:46.070203 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77da122-e4b5-4c90-a2ee-e038bcb28965 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:46.197531 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77da122-e4b5-4c90-a2ee-e038bcb28965 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:46.240231 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d86d5a-7974-46ac-8701-03c27257d462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:46.241107 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d86d5a-7974-46ac-8701-03c27257d462 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:46.241428 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d86d5a-7974-46ac-8701-03c27257d462 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:46.241707 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d86d5a-7974-46ac-8701-03c27257d462 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:46.242469 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d86d5a-7974-46ac-8701-03c27257d462 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:46.242817 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d86d5a-7974-46ac-8701-03c27257d462 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:46.246191 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d86d5a-7974-46ac-8701-03c27257d462 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:46.250567 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b22fdb-2020-4294-8260-8a4504e452e3 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:46.267000 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 389/792] 10.222.0.43 () {56 vars in 1149 bytes} [Tue Sep 6 16:03:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:46.274931 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d86d5a-7974-46ac-8701-03c27257d462 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:46.323377 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:46.336861 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efcd41a-263b-41d3-86b4-92a9449a5d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:46.338471 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efcd41a-263b-41d3-86b4-92a9449a5d18 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:46.338801 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efcd41a-263b-41d3-86b4-92a9449a5d18 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:46.339075 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efcd41a-263b-41d3-86b4-92a9449a5d18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:46.341557 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:46.357963 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efcd41a-263b-41d3-86b4-92a9449a5d18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:46.359458 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efcd41a-263b-41d3-86b4-92a9449a5d18 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:46.377169 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-6efcd41a-263b-41d3-86b4-92a9449a5d18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:46.377169 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:46.377169 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:46.377169 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:46.377169 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:46.377169 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:46.377169 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:46.377169 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:46.377169 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:46.377169 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:46.377169 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:46.377169 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:46.377169 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:03:46.377169 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:03:46.377169 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:03:46.377169 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:03:46.378344 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:46.378344 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:46.378344 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:46.378344 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:46.378344 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:46.378344 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:46.378344 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:46.378344 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:46.378344 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:46.378344 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:46.378344 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:46.378344 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:46.378344 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:46.378344 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:46.378344 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:46.378344 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:46.378344 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:03:46.379214 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:03:46.379214 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:46.379214 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:46.380709 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 404/793] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:03:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:46.382117 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d62e108-3458-4b85-8f36-7602ac241961 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:46.382391 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d62e108-3458-4b85-8f36-7602ac241961 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:46.382655 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d62e108-3458-4b85-8f36-7602ac241961 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:46.383992 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 405/794] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:03:46 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:03:46.389544 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b18eb0c-8e01-4c69-84a8-aec5b555bcbf None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:46.389831 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b18eb0c-8e01-4c69-84a8-aec5b555bcbf None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:46.390099 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b18eb0c-8e01-4c69-84a8-aec5b555bcbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:46.408294 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-1b18eb0c-8e01-4c69-84a8-aec5b555bcbf None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:03:46.409648 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-1b18eb0c-8e01-4c69-84a8-aec5b555bcbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:03:46.424868 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d86d5a-7974-46ac-8701-03c27257d462 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:46.463310 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a0e22f-4239-4f43-80d7-c4ad8fa1cf8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:46.464002 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0e22f-4239-4f43-80d7-c4ad8fa1cf8f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:46.464273 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0e22f-4239-4f43-80d7-c4ad8fa1cf8f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:46.464537 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0e22f-4239-4f43-80d7-c4ad8fa1cf8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:46.476229 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a0e22f-4239-4f43-80d7-c4ad8fa1cf8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:46.477741 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a0e22f-4239-4f43-80d7-c4ad8fa1cf8f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:46.479566 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 406/795] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:03:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 16:03:46.480471 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b18eb0c-8e01-4c69-84a8-aec5b555bcbf None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:46.502478 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-a1a0e22f-4239-4f43-80d7-c4ad8fa1cf8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:46.502478 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:46.502478 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:46.502478 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:46.502478 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:46.502478 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:46.502478 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:46.502478 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:46.502478 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:46.502478 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:46.502478 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:46.502478 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:46.502478 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:03:46.502478 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:03:46.502478 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:03:46.502478 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:03:46.503546 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:46.503546 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:46.503546 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:46.503546 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:46.503546 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:46.503546 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:46.503546 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:46.503546 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:46.503546 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:46.503546 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:46.503546 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:46.503546 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:46.503546 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:46.503546 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:46.503546 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:46.503546 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:46.503546 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:03:46.504459 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:03:46.504459 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:46.504459 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:46.504459 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 390/796] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:03:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:46.509869 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a0e22f-4239-4f43-80d7-c4ad8fa1cf8f None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:46.607008 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:46.621644 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a0e22f-4239-4f43-80d7-c4ad8fa1cf8f None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:46.658013 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6db08ec-52c4-49f8-886d-1e278c78a6c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:46.658750 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6db08ec-52c4-49f8-886d-1e278c78a6c2 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:46.659090 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6db08ec-52c4-49f8-886d-1e278c78a6c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:46.659376 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6db08ec-52c4-49f8-886d-1e278c78a6c2 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:46.660133 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6db08ec-52c4-49f8-886d-1e278c78a6c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:46.660487 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6db08ec-52c4-49f8-886d-1e278c78a6c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:46.661937 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6db08ec-52c4-49f8-886d-1e278c78a6c2 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:46.678765 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 391/797] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:03:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:03:46.680300 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6db08ec-52c4-49f8-886d-1e278c78a6c2 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:46.714984 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b18eb0c-8e01-4c69-84a8-aec5b555bcbf None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:46.756321 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:46.786479 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6db08ec-52c4-49f8-886d-1e278c78a6c2 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:46.786977 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b32816-44dc-4bd0-9de7-f3239decbc22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:46.787603 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b32816-44dc-4bd0-9de7-f3239decbc22 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:46.787876 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b32816-44dc-4bd0-9de7-f3239decbc22 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:46.788146 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b32816-44dc-4bd0-9de7-f3239decbc22 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:46.788945 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b32816-44dc-4bd0-9de7-f3239decbc22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:46.789249 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b32816-44dc-4bd0-9de7-f3239decbc22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:46.790726 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b32816-44dc-4bd0-9de7-f3239decbc22 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:46.810709 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 407/798] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:03:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:46.811560 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b32816-44dc-4bd0-9de7-f3239decbc22 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:46.823987 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b092578-8a80-4a55-ba4f-b5952cfe231a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:46.824874 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b092578-8a80-4a55-ba4f-b5952cfe231a None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:46.825286 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b092578-8a80-4a55-ba4f-b5952cfe231a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:46.825790 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b092578-8a80-4a55-ba4f-b5952cfe231a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:46.833967 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b092578-8a80-4a55-ba4f-b5952cfe231a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:46.835731 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b092578-8a80-4a55-ba4f-b5952cfe231a None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:46.848060 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-9b092578-8a80-4a55-ba4f-b5952cfe231a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:03:46.848060 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:46.848060 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:46.848060 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:46.848060 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:46.848060 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:46.848060 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:46.848060 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:46.848060 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:46.848060 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:46.848060 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:46.848060 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:46.848060 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:03:46.848060 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:03:46.848060 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:03:46.849193 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:03:46.849193 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:46.849193 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:46.849193 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:46.849193 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:46.849193 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:46.849193 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:46.849193 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:46.849193 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:46.849193 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:46.849193 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:46.849193 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:46.849193 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:46.849193 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:46.849193 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:46.849193 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:46.849193 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:46.850071 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:03:46.850071 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:03:46.850071 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:03:46.850071 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:03:46.850071 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:03:46.850071 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:03:46.850071 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:03:46.850071 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:46.855394 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 392/799] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 6 16:03:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:03:46.855394 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b092578-8a80-4a55-ba4f-b5952cfe231a None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:46.868240 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:46.925071 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b32816-44dc-4bd0-9de7-f3239decbc22 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:46.960907 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b092578-8a80-4a55-ba4f-b5952cfe231a None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:46.963002 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74364d09-b9f2-4585-899d-b02dbf136051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:46.963742 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74364d09-b9f2-4585-899d-b02dbf136051 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:46.964024 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74364d09-b9f2-4585-899d-b02dbf136051 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:46.964295 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74364d09-b9f2-4585-899d-b02dbf136051 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:46.964912 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74364d09-b9f2-4585-899d-b02dbf136051 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:46.966288 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74364d09-b9f2-4585-899d-b02dbf136051 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:46.976558 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-74364d09-b9f2-4585-899d-b02dbf136051 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 16:03:46.976558 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:46.976558 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 16:03:46.976558 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 16:03:46.976558 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 16:03:46.976558 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 16:03:46.976558 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 16:03:46.976558 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:03:46.976558 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 16:03:46.976558 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:46.976558 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 16:03:46.976558 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:46.976558 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:46.976558 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:46.976558 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:46.976558 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:46.976558 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:46.977657 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:46.977657 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:46.977657 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:46.977657 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:46.977657 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:46.977657 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:46.977657 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 16:03:46.977657 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 16:03:46.977657 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 16:03:46.977657 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 16:03:46.977657 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:46.977657 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:46.977657 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:46.977657 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:46.977657 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:46.977657 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:46.978498 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:46.978498 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:46.978498 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:46.978498 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:46.978498 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:46.978498 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:46.978498 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:46.978498 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:46.978498 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:46.978498 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:46.978498 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 16:03:46.978498 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:03:46.978498 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 16:03:46.978498 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:46.978498 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 408/800] 10.222.0.43 () {56 vars in 1129 bytes} [Tue Sep 6 16:03:46 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:46.979277 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74364d09-b9f2-4585-899d-b02dbf136051 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:46.998640 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3554774d-384b-4cd5-9a60-e46cdef8af72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:46.999277 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3554774d-384b-4cd5-9a60-e46cdef8af72 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:46.999648 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3554774d-384b-4cd5-9a60-e46cdef8af72 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:46.999918 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3554774d-384b-4cd5-9a60-e46cdef8af72 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:47.008196 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3554774d-384b-4cd5-9a60-e46cdef8af72 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:47.009769 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3554774d-384b-4cd5-9a60-e46cdef8af72 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:47.021526 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-3554774d-384b-4cd5-9a60-e46cdef8af72 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 06 16:03:47.021526 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:47.021526 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:47.021526 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:47.021526 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:47.021526 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:47.021526 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:47.021526 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:47.021526 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:47.021526 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:47.021526 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:47.021526 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:47.021526 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:03:47.021526 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:03:47.021526 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:03:47.021526 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:03:47.023554 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:47.023554 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:47.023554 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:03:47.023554 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:03:47.023554 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:03:47.023554 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:03:47.023554 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:47.023554 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:47.023554 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:47.023554 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:47.023554 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:47.023554 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:47.023554 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:47.023554 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:47.023554 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:47.023554 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:47.023554 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:47.023554 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:47.025734 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:47.025734 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:47.025734 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:03:47.025734 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:03:47.025734 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:03:47.025734 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:03:47.025734 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:03:47.025734 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:03:47.025734 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 06 16:03:47.025734 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:47.025734 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 393/801] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 6 16:03:46 2022] GET /identity/v3/users/swift => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:47.025734 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3554774d-384b-4cd5-9a60-e46cdef8af72 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:47.026659 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:47.092897 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74364d09-b9f2-4585-899d-b02dbf136051 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:47.095811 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:47.128566 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3554774d-384b-4cd5-9a60-e46cdef8af72 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:47.130342 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d7c665-173a-457f-9a18-c73d5d2523d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:47.131198 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d7c665-173a-457f-9a18-c73d5d2523d3 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:47.131623 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d7c665-173a-457f-9a18-c73d5d2523d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:47.132041 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d7c665-173a-457f-9a18-c73d5d2523d3 None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:47.132893 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d7c665-173a-457f-9a18-c73d5d2523d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:47.133328 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d7c665-173a-457f-9a18-c73d5d2523d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:47.134880 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d7c665-173a-457f-9a18-c73d5d2523d3 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:47.147734 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 409/802] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 6 16:03:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:47.148785 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d7c665-173a-457f-9a18-c73d5d2523d3 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:47.175091 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae84ab7-706b-416b-8e02-66b76d381479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:47.176030 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae84ab7-706b-416b-8e02-66b76d381479 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:47.176726 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae84ab7-706b-416b-8e02-66b76d381479 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:47.177417 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae84ab7-706b-416b-8e02-66b76d381479 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:47.178273 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae84ab7-706b-416b-8e02-66b76d381479 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:47.178832 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae84ab7-706b-416b-8e02-66b76d381479 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:47.180928 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae84ab7-706b-416b-8e02-66b76d381479 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:47.192422 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 394/803] 10.222.0.43 () {56 vars in 1124 bytes} [Tue Sep 6 16:03:47 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:47.193429 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae84ab7-706b-416b-8e02-66b76d381479 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:47.252220 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:47.256288 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d7c665-173a-457f-9a18-c73d5d2523d3 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:47.297253 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae14ef8-8a48-4e43-a552-242c94e62e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:47.297915 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae14ef8-8a48-4e43-a552-242c94e62e05 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:47.298642 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae14ef8-8a48-4e43-a552-242c94e62e05 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:47.298922 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae14ef8-8a48-4e43-a552-242c94e62e05 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:47.299923 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae14ef8-8a48-4e43-a552-242c94e62e05 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:47.300345 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae14ef8-8a48-4e43-a552-242c94e62e05 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:47.303403 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae14ef8-8a48-4e43-a552-242c94e62e05 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:47.320607 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae84ab7-706b-416b-8e02-66b76d381479 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:47.328840 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 410/804] 10.222.0.43 () {56 vars in 1135 bytes} [Tue Sep 6 16:03:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:47.330249 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae14ef8-8a48-4e43-a552-242c94e62e05 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:47.361464 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5032f62a-da55-4341-b205-dbb6a6abfb7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:47.362443 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5032f62a-da55-4341-b205-dbb6a6abfb7b None admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:47.362986 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5032f62a-da55-4341-b205-dbb6a6abfb7b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:47.363628 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5032f62a-da55-4341-b205-dbb6a6abfb7b None admin] PATH_INFO: `/v3/projects/128d210e8dad48ac921ed4cb1d444ceb/users/d865006413b24ec3922619311f81f353/roles/31321627b02c4ff188f8fe1521ce3181` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:47.366605 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:47.384938 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5032f62a-da55-4341-b205-dbb6a6abfb7b None admin] RBAC: Authorizing `identity:create_grant(project_id=128d210e8dad48ac921ed4cb1d444ceb, user_id=d865006413b24ec3922619311f81f353, role_id=31321627b02c4ff188f8fe1521ce3181)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:47.386767 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5032f62a-da55-4341-b205-dbb6a6abfb7b None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:47.411189 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 395/805] 10.222.0.43 () {58 vars in 1439 bytes} [Tue Sep 6 16:03:47 2022] PUT /identity/v3/projects/128d210e8dad48ac921ed4cb1d444ceb/users/d865006413b24ec3922619311f81f353/roles/31321627b02c4ff188f8fe1521ce3181 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:03:47.411189 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5032f62a-da55-4341-b205-dbb6a6abfb7b None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:47.537003 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae14ef8-8a48-4e43-a552-242c94e62e05 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:47.550740 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5032f62a-da55-4341-b205-dbb6a6abfb7b None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:47.592426 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ef8b2d-5695-4012-9b10-140b9136961d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:47.593423 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef8b2d-5695-4012-9b10-140b9136961d None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:47.594060 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef8b2d-5695-4012-9b10-140b9136961d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:47.594520 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef8b2d-5695-4012-9b10-140b9136961d None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:47.595237 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d4d6e4-bfac-425b-8a2f-24e1b34395bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:47.595572 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ef8b2d-5695-4012-9b10-140b9136961d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:47.595881 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d4d6e4-bfac-425b-8a2f-24e1b34395bb None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:47.596160 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d4d6e4-bfac-425b-8a2f-24e1b34395bb None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:47.596446 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d4d6e4-bfac-425b-8a2f-24e1b34395bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:47.596711 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ef8b2d-5695-4012-9b10-140b9136961d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:47.598285 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ef8b2d-5695-4012-9b10-140b9136961d None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:47.605148 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 411/806] 10.222.0.43 () {56 vars in 1124 bytes} [Tue Sep 6 16:03:47 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:47.610365 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ef8b2d-5695-4012-9b10-140b9136961d None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:47.610779 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d4d6e4-bfac-425b-8a2f-24e1b34395bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:47.612370 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d4d6e4-bfac-425b-8a2f-24e1b34395bb None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:47.632218 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-20d4d6e4-bfac-425b-8a2f-24e1b34395bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:47.632218 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:47.632218 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:47.632218 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:47.632218 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:47.632218 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:47.632218 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:47.632218 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:47.632218 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:47.632218 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:47.632218 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:47.632218 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:47.632218 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:03:47.632218 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:03:47.632218 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:03:47.632218 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:03:47.633289 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:47.633289 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:47.633289 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:47.633289 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:47.633289 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:47.633289 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:47.633289 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:47.633289 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:47.633289 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:47.633289 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:47.633289 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:47.633289 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:47.633289 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:47.633289 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:47.633289 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:47.633289 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:47.633289 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:03:47.634195 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:03:47.634195 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:47.634195 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:47.634195 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 396/807] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:03:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:47.639677 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d4d6e4-bfac-425b-8a2f-24e1b34395bb None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:47.644214 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:47.690690 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:47.748298 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ef8b2d-5695-4012-9b10-140b9136961d None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:47.767055 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d4d6e4-bfac-425b-8a2f-24e1b34395bb None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:47.793990 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571f84ed-ed08-4906-8c27-6a8876377ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:47.794613 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571f84ed-ed08-4906-8c27-6a8876377ae0 None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:47.795129 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571f84ed-ed08-4906-8c27-6a8876377ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:47.795408 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571f84ed-ed08-4906-8c27-6a8876377ae0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:47.796500 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571f84ed-ed08-4906-8c27-6a8876377ae0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:47.798176 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571f84ed-ed08-4906-8c27-6a8876377ae0 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:47.804602 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0872685b-0efc-4f55-a01b-2e927d79d68f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:47.805562 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0872685b-0efc-4f55-a01b-2e927d79d68f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:47.806050 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0872685b-0efc-4f55-a01b-2e927d79d68f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:47.806519 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0872685b-0efc-4f55-a01b-2e927d79d68f None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:47.807469 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0872685b-0efc-4f55-a01b-2e927d79d68f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:47.808012 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0872685b-0efc-4f55-a01b-2e927d79d68f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:47.809647 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0872685b-0efc-4f55-a01b-2e927d79d68f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:47.824976 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 412/808] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:03:47 2022] POST /identity/v3/endpoints => generated 346 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:03:47.825438 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 397/809] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:03:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:47.831519 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571f84ed-ed08-4906-8c27-6a8876377ae0 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:47.932605 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:47.974767 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571f84ed-ed08-4906-8c27-6a8876377ae0 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:47.981305 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ce7e2-d34e-4029-8cb3-d1961b402207 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:47.981646 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ce7e2-d34e-4029-8cb3-d1961b402207 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:47.981853 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ce7e2-d34e-4029-8cb3-d1961b402207 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:47.983504 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 398/810] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:03:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:03:47.989352 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b857ea5b-1d9c-4456-b069-51dc43f58e8c None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:47.989642 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b857ea5b-1d9c-4456-b069-51dc43f58e8c None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:47.989907 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b857ea5b-1d9c-4456-b069-51dc43f58e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:48.009374 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-b857ea5b-1d9c-4456-b069-51dc43f58e8c None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:03:48.010594 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b857ea5b-1d9c-4456-b069-51dc43f58e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:03:48.011840 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce6e9a1-11cc-43c3-a374-cd67a021fa9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:48.012510 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce6e9a1-11cc-43c3-a374-cd67a021fa9a None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:48.012818 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce6e9a1-11cc-43c3-a374-cd67a021fa9a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:48.013104 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce6e9a1-11cc-43c3-a374-cd67a021fa9a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:48.020877 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce6e9a1-11cc-43c3-a374-cd67a021fa9a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:48.022472 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce6e9a1-11cc-43c3-a374-cd67a021fa9a None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:48.034989 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-bce6e9a1-11cc-43c3-a374-cd67a021fa9a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:03:48.034989 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:48.034989 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:48.034989 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:48.034989 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:48.034989 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:48.034989 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:48.034989 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:48.034989 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:48.034989 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:48.034989 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:48.034989 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:48.034989 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:03:48.034989 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:03:48.034989 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:03:48.035979 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:03:48.035979 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:48.035979 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:48.035979 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:48.035979 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:48.035979 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:48.035979 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:48.035979 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:48.035979 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:48.035979 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:48.035979 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:48.035979 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:48.035979 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:48.035979 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:48.035979 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:48.035979 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:48.035979 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:48.037332 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:03:48.037332 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:03:48.037332 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:03:48.037332 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:03:48.037332 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:03:48.037332 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:03:48.037332 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:03:48.037332 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:48.037332 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 413/811] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 6 16:03:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:48.044156 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce6e9a1-11cc-43c3-a374-cd67a021fa9a None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:48.128320 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 399/812] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:03:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 16:03:48.166095 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce6e9a1-11cc-43c3-a374-cd67a021fa9a None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:48.200275 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef6c0c3-ff34-41e2-abdc-02143b139efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:48.200951 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef6c0c3-ff34-41e2-abdc-02143b139efd None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:48.201289 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef6c0c3-ff34-41e2-abdc-02143b139efd None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:48.201629 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef6c0c3-ff34-41e2-abdc-02143b139efd None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:48.202395 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef6c0c3-ff34-41e2-abdc-02143b139efd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:48.202774 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef6c0c3-ff34-41e2-abdc-02143b139efd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:48.204654 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef6c0c3-ff34-41e2-abdc-02143b139efd None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:48.206418 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:48.218014 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 414/813] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 6 16:03:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:03:48.223687 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b857ea5b-1d9c-4456-b069-51dc43f58e8c None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:48.254109 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:48.330567 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b857ea5b-1d9c-4456-b069-51dc43f58e8c None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:48.379673 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d655d2-8b84-46c0-8f52-8bfb8d1e24d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:48.380340 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d655d2-8b84-46c0-8f52-8bfb8d1e24d2 None admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:48.380617 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d655d2-8b84-46c0-8f52-8bfb8d1e24d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:48.380922 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d655d2-8b84-46c0-8f52-8bfb8d1e24d2 None admin] PATH_INFO: `/v3/projects/128d210e8dad48ac921ed4cb1d444ceb/users/312026c080b64aff9dd483d3591288a6/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:48.386041 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d655d2-8b84-46c0-8f52-8bfb8d1e24d2 None admin] RBAC: Authorizing `identity:create_grant(project_id=128d210e8dad48ac921ed4cb1d444ceb, user_id=312026c080b64aff9dd483d3591288a6, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:48.387562 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d655d2-8b84-46c0-8f52-8bfb8d1e24d2 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:48.410108 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 400/814] 10.222.0.43 () {58 vars in 1439 bytes} [Tue Sep 6 16:03:48 2022] PUT /identity/v3/projects/128d210e8dad48ac921ed4cb1d444ceb/users/312026c080b64aff9dd483d3591288a6/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:03:48.486640 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23097c2e-f1bd-4a10-946f-43765cd69c4d None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:48.486640 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23097c2e-f1bd-4a10-946f-43765cd69c4d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:48.486640 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23097c2e-f1bd-4a10-946f-43765cd69c4d None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:48.487415 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 415/815] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:03:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:03:48.493468 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2687ce-4515-42a4-9d01-8ff8bff0a324 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:48.494073 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2687ce-4515-42a4-9d01-8ff8bff0a324 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:48.494545 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2687ce-4515-42a4-9d01-8ff8bff0a324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:48.506609 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23097c2e-f1bd-4a10-946f-43765cd69c4d None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:48.517629 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-3c2687ce-4515-42a4-9d01-8ff8bff0a324 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:03:48.519380 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3c2687ce-4515-42a4-9d01-8ff8bff0a324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:03:48.615200 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 401/816] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:03:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:03:48.634639 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23097c2e-f1bd-4a10-946f-43765cd69c4d None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:48.686023 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e36604-8b06-4e93-a73f-2b7dbe984af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:48.686720 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e36604-8b06-4e93-a73f-2b7dbe984af6 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:48.686993 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e36604-8b06-4e93-a73f-2b7dbe984af6 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:48.687290 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e36604-8b06-4e93-a73f-2b7dbe984af6 None admin] PATH_INFO: `/v3/services/network` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:48.687887 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e36604-8b06-4e93-a73f-2b7dbe984af6 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:48.689511 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e36604-8b06-4e93-a73f-2b7dbe984af6 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:48.703249 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-f8e36604-8b06-4e93-a73f-2b7dbe984af6 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 16:03:48.703249 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:48.703249 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 16:03:48.703249 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 16:03:48.703249 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 16:03:48.703249 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 16:03:48.703249 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 16:03:48.703249 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:03:48.703249 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 16:03:48.703249 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:48.703249 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 16:03:48.703249 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:48.703249 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:48.703249 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:48.703249 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:48.703249 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:48.703249 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:48.704654 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:48.704654 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:48.704654 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:48.704654 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:48.704654 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:48.704654 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:48.704654 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 16:03:48.704654 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 16:03:48.704654 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 16:03:48.704654 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 16:03:48.704654 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:48.704654 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:48.704654 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:48.704654 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:48.704654 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:48.704654 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:48.706167 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:48.706167 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:48.706167 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:48.706167 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:48.706167 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:48.706167 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:48.706167 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:48.706167 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:48.706167 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:48.706167 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:48.706167 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 16:03:48.706167 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:03:48.706167 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 16:03:48.706167 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:48.706167 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 416/817] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 6 16:03:48 2022] GET /identity/v3/services/network => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:48.710699 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2687ce-4515-42a4-9d01-8ff8bff0a324 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:48.733713 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:48.831539 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2687ce-4515-42a4-9d01-8ff8bff0a324 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:48.871762 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5a9d28-8f67-4c3b-ac25-9c9e09aca70b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:48.872783 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5a9d28-8f67-4c3b-ac25-9c9e09aca70b None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:48.873291 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5a9d28-8f67-4c3b-ac25-9c9e09aca70b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:48.873780 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5a9d28-8f67-4c3b-ac25-9c9e09aca70b None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:48.874626 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5a9d28-8f67-4c3b-ac25-9c9e09aca70b None admin] RBAC: Adding query filter params (name=network) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:48.875128 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5a9d28-8f67-4c3b-ac25-9c9e09aca70b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:48.876825 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5a9d28-8f67-4c3b-ac25-9c9e09aca70b None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:48.885348 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 402/818] 10.222.0.43 () {56 vars in 1112 bytes} [Tue Sep 6 16:03:48 2022] GET /identity/v3/services?name=network => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:48.891121 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e36604-8b06-4e93-a73f-2b7dbe984af6 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:48.952692 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:48.986855 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5a9d28-8f67-4c3b-ac25-9c9e09aca70b None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:49.005882 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e36604-8b06-4e93-a73f-2b7dbe984af6 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:49.071379 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56c0287-67ff-483a-aefe-4cae3b8fdd15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:49.072407 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56c0287-67ff-483a-aefe-4cae3b8fdd15 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:49.072921 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56c0287-67ff-483a-aefe-4cae3b8fdd15 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:49.073390 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56c0287-67ff-483a-aefe-4cae3b8fdd15 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:49.074231 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56c0287-67ff-483a-aefe-4cae3b8fdd15 None admin] RBAC: Adding query filter params (type=network) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:49.074744 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56c0287-67ff-483a-aefe-4cae3b8fdd15 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:49.076525 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56c0287-67ff-483a-aefe-4cae3b8fdd15 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:49.089101 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 417/819] 10.222.0.43 () {56 vars in 1112 bytes} [Tue Sep 6 16:03:48 2022] GET /identity/v3/services?type=network => generated 350 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:49.093273 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56c0287-67ff-483a-aefe-4cae3b8fdd15 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:49.158448 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:49.198802 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5a9d28-8f67-4c3b-ac25-9c9e09aca70b None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:49.243037 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56c0287-67ff-483a-aefe-4cae3b8fdd15 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:49.258757 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16a4bc2-5202-49d0-b9f5-1faf42ee98e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:49.259755 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a4bc2-5202-49d0-b9f5-1faf42ee98e5 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:49.259755 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a4bc2-5202-49d0-b9f5-1faf42ee98e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:49.259974 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a4bc2-5202-49d0-b9f5-1faf42ee98e5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:49.283483 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:49.289372 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21be2785-9cf5-43da-9f3c-ef5a4bf7a33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:49.290304 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be2785-9cf5-43da-9f3c-ef5a4bf7a33a None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:49.290797 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be2785-9cf5-43da-9f3c-ef5a4bf7a33a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:49.291275 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be2785-9cf5-43da-9f3c-ef5a4bf7a33a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:49.292144 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21be2785-9cf5-43da-9f3c-ef5a4bf7a33a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:49.293797 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21be2785-9cf5-43da-9f3c-ef5a4bf7a33a None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:49.294625 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16a4bc2-5202-49d0-b9f5-1faf42ee98e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:03:49.299828 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-a16a4bc2-5202-49d0-b9f5-1faf42ee98e5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:03:49.299828 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:49.299828 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:49.299828 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:49.299828 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:49.299828 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:49.299828 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:49.299828 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:49.299828 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:49.299828 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:49.299828 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:49.299828 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:49.299828 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:03:49.299828 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:03:49.299828 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:03:49.299828 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:03:49.302097 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:03:49.302097 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:03:49.302097 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:03:49.302097 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:03:49.302097 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:49.302097 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:49.302097 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:49.302097 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:49.302097 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:49.302097 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:49.302097 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:49.302097 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:49.302097 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:49.302097 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:49.302097 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:49.302097 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:49.303672 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:49.303672 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:49.303672 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:49.303672 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:49.303672 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:03:49.303672 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:03:49.303672 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:03:49.303672 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:03:49.303672 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:03:49.303672 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:03:49.303672 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:03:49.303672 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:49.303672 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 403/820] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 6 16:03:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:03:49.309414 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16a4bc2-5202-49d0-b9f5-1faf42ee98e5 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:49.325225 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 418/821] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:03:49 2022] POST /identity/v3/endpoints => generated 337 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:03:49.431484 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16a4bc2-5202-49d0-b9f5-1faf42ee98e5 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:49.484083 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c694d500-4f16-40b2-ae86-0f709a0e3a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:49.484723 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c694d500-4f16-40b2-ae86-0f709a0e3a2f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:49.485025 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c694d500-4f16-40b2-ae86-0f709a0e3a2f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:49.485302 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c694d500-4f16-40b2-ae86-0f709a0e3a2f None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:49.485980 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c694d500-4f16-40b2-ae86-0f709a0e3a2f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:49.486280 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c694d500-4f16-40b2-ae86-0f709a0e3a2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:49.487738 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c694d500-4f16-40b2-ae86-0f709a0e3a2f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:49.493224 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:49.501135 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 404/822] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 6 16:03:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:03:49.505964 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21be2785-9cf5-43da-9f3c-ef5a4bf7a33a None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:49.624424 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21be2785-9cf5-43da-9f3c-ef5a4bf7a33a None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:49.659640 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30241957-8bef-4b57-940f-b94ace290324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:49.660635 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30241957-8bef-4b57-940f-b94ace290324 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:49.661140 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30241957-8bef-4b57-940f-b94ace290324 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:49.661602 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30241957-8bef-4b57-940f-b94ace290324 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:49.672725 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30241957-8bef-4b57-940f-b94ace290324 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:49.674689 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30241957-8bef-4b57-940f-b94ace290324 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:49.690512 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-30241957-8bef-4b57-940f-b94ace290324 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:49.690512 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:49.690512 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:49.690512 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:49.690512 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:49.690512 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:49.690512 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:49.690512 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:49.690512 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:49.690512 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:49.690512 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:49.690512 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:49.690512 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:03:49.690512 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:03:49.690512 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:03:49.690512 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:03:49.691452 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:49.691452 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:49.691452 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:49.691452 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:49.691452 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:49.691452 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:49.691452 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:49.691452 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:49.691452 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:49.691452 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:49.691452 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:49.691452 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:49.691452 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:49.691452 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:49.691452 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:49.691452 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:49.691452 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:03:49.692385 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:03:49.692385 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:49.692385 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:49.694748 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 419/823] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:03:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:49.700707 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c694d500-4f16-40b2-ae86-0f709a0e3a2f None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:49.777022 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d79407-b2bd-49e9-9529-1e446df75ecd None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:49.777701 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d79407-b2bd-49e9-9529-1e446df75ecd None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:49.778236 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d79407-b2bd-49e9-9529-1e446df75ecd None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:49.780092 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 420/824] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:03:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 16:03:49.785848 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a81b8-6ee4-4402-a008-707ccc473c39 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:49.786350 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a81b8-6ee4-4402-a008-707ccc473c39 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:49.786817 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a81b8-6ee4-4402-a008-707ccc473c39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:49.813137 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-277a81b8-6ee4-4402-a008-707ccc473c39 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:03:49.814954 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-277a81b8-6ee4-4402-a008-707ccc473c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:03:49.843513 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c694d500-4f16-40b2-ae86-0f709a0e3a2f None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:49.851372 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:49.898739 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9840f5-a5a3-4530-b20f-985f51f129f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:49.900555 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9840f5-a5a3-4530-b20f-985f51f129f3 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:49.900788 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9840f5-a5a3-4530-b20f-985f51f129f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:49.901075 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9840f5-a5a3-4530-b20f-985f51f129f3 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:49.901723 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9840f5-a5a3-4530-b20f-985f51f129f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:49.902021 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9840f5-a5a3-4530-b20f-985f51f129f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:49.903408 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9840f5-a5a3-4530-b20f-985f51f129f3 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:49.914759 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 421/825] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:03:49 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:03:49.923198 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 405/826] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:03:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:49.933890 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277a81b8-6ee4-4402-a008-707ccc473c39 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:50.043179 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277a81b8-6ee4-4402-a008-707ccc473c39 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:50.079486 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b8895e-d74f-4408-b59e-12450e222d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:50.080476 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b8895e-d74f-4408-b59e-12450e222d6f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:50.081016 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b8895e-d74f-4408-b59e-12450e222d6f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:50.081506 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b8895e-d74f-4408-b59e-12450e222d6f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:50.089053 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b8895e-d74f-4408-b59e-12450e222d6f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:50.090706 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b8895e-d74f-4408-b59e-12450e222d6f None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:50.102585 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-03b8895e-d74f-4408-b59e-12450e222d6f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 16:03:50.102585 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:50.102585 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:50.102585 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:50.102585 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:50.102585 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:50.102585 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:50.102585 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:50.102585 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:50.102585 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:50.102585 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:50.102585 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:50.102585 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:03:50.102585 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:03:50.102585 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:03:50.102585 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:03:50.103538 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:50.103538 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:50.103538 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:03:50.103538 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:03:50.103538 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:03:50.103538 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:03:50.103538 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:50.103538 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:50.103538 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:50.103538 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:50.103538 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:50.103538 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:50.103538 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:50.103538 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:50.103538 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:50.103538 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:50.103538 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:50.103538 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:50.104485 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:50.104485 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:50.104485 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:03:50.104485 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:03:50.104485 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:03:50.104485 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:03:50.104485 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:03:50.104485 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:03:50.104485 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 16:03:50.104485 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:50.106570 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 422/827] 10.222.0.43 () {56 vars in 1093 bytes} [Tue Sep 6 16:03:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:50.113913 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9840f5-a5a3-4530-b20f-985f51f129f3 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:50.177794 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:50.184427 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b8895e-d74f-4408-b59e-12450e222d6f None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:50.212527 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:50.225158 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9840f5-a5a3-4530-b20f-985f51f129f3 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:50.262961 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb562a4-cd75-4882-a95c-8b53a0732a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:50.263585 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb562a4-cd75-4882-a95c-8b53a0732a82 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:50.263862 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb562a4-cd75-4882-a95c-8b53a0732a82 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:50.264129 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb562a4-cd75-4882-a95c-8b53a0732a82 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:50.264893 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb562a4-cd75-4882-a95c-8b53a0732a82 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:50.265195 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb562a4-cd75-4882-a95c-8b53a0732a82 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:50.266636 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb562a4-cd75-4882-a95c-8b53a0732a82 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:50.287813 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 406/828] 10.222.0.43 () {56 vars in 1133 bytes} [Tue Sep 6 16:03:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:50.288923 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268cc448-40a8-4689-b501-adac6a70b50c None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:50.289205 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268cc448-40a8-4689-b501-adac6a70b50c None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:50.289473 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268cc448-40a8-4689-b501-adac6a70b50c None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:50.290829 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 407/829] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:03:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:03:50.295490 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268cc448-40a8-4689-b501-adac6a70b50c None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:50.308430 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b8895e-d74f-4408-b59e-12450e222d6f None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:50.344324 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307d713e-94ac-40fe-9303-cbf142915795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:50.345003 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d713e-94ac-40fe-9303-cbf142915795 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:50.345285 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d713e-94ac-40fe-9303-cbf142915795 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:50.345554 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d713e-94ac-40fe-9303-cbf142915795 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:50.358543 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307d713e-94ac-40fe-9303-cbf142915795 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:03:50.363850 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-307d713e-94ac-40fe-9303-cbf142915795 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:03:50.363850 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:50.363850 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:50.363850 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:50.363850 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:50.363850 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:50.363850 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:50.363850 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:50.363850 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:50.363850 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:50.363850 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:50.363850 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:50.363850 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:03:50.363850 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:03:50.363850 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:03:50.363850 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:03:50.364851 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:03:50.364851 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:03:50.364851 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:03:50.364851 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:03:50.364851 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:50.364851 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:50.364851 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:50.364851 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:50.364851 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:50.364851 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:50.364851 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:50.364851 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:50.364851 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:50.364851 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:50.364851 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:50.364851 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:50.365700 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:50.365700 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:50.365700 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:50.365700 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:50.365700 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:03:50.365700 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:03:50.365700 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:03:50.365700 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:03:50.365700 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:03:50.365700 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:03:50.365700 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:03:50.365700 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:50.365700 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 423/830] 10.222.0.43 () {56 vars in 1102 bytes} [Tue Sep 6 16:03:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:03:50.366911 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7090f794-7453-447c-b8ac-318a59774f0c None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:50.366911 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7090f794-7453-447c-b8ac-318a59774f0c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:50.366911 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7090f794-7453-447c-b8ac-318a59774f0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:50.386120 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-7090f794-7453-447c-b8ac-318a59774f0c None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:03:50.387311 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7090f794-7453-447c-b8ac-318a59774f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:03:50.406745 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268cc448-40a8-4689-b501-adac6a70b50c None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:50.423943 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:50.443003 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd7bad6-5d64-42e7-9ed4-bc8724bca575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:50.443617 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd7bad6-5d64-42e7-9ed4-bc8724bca575 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:50.443894 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd7bad6-5d64-42e7-9ed4-bc8724bca575 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:50.444163 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd7bad6-5d64-42e7-9ed4-bc8724bca575 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:50.460540 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 424/831] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:03:50 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:03:50.461623 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7090f794-7453-447c-b8ac-318a59774f0c None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:50.481832 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:50.524158 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd7bad6-5d64-42e7-9ed4-bc8724bca575 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:50.525703 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd7bad6-5d64-42e7-9ed4-bc8724bca575 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:50.604961 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7090f794-7453-447c-b8ac-318a59774f0c None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:50.633184 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 408/832] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 6 16:03:50 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 16:03:50.634810 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd7bad6-5d64-42e7-9ed4-bc8724bca575 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:50.651391 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e399bac-5f33-484b-9cef-b067026bde31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:50.652361 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e399bac-5f33-484b-9cef-b067026bde31 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:50.652959 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:50.653778 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e399bac-5f33-484b-9cef-b067026bde31 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:50.654503 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e399bac-5f33-484b-9cef-b067026bde31 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:50.655329 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e399bac-5f33-484b-9cef-b067026bde31 None admin] RBAC: Adding query filter params (name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:50.655827 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e399bac-5f33-484b-9cef-b067026bde31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:50.657914 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e399bac-5f33-484b-9cef-b067026bde31 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:50.670652 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 425/833] 10.222.0.43 () {56 vars in 1103 bytes} [Tue Sep 6 16:03:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:50.677114 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e399bac-5f33-484b-9cef-b067026bde31 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:50.745489 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd7bad6-5d64-42e7-9ed4-bc8724bca575 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:50.767393 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:50.786071 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3260c67-8c14-4b5c-91b7-a8ba8fa19b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:50.787133 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3260c67-8c14-4b5c-91b7-a8ba8fa19b3c None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:50.787412 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3260c67-8c14-4b5c-91b7-a8ba8fa19b3c None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:50.787861 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3260c67-8c14-4b5c-91b7-a8ba8fa19b3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:50.798178 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e399bac-5f33-484b-9cef-b067026bde31 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:50.798473 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3260c67-8c14-4b5c-91b7-a8ba8fa19b3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:50.799983 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3260c67-8c14-4b5c-91b7-a8ba8fa19b3c None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:50.815003 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-c3260c67-8c14-4b5c-91b7-a8ba8fa19b3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:50.815003 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:50.815003 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:50.815003 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:50.815003 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:50.815003 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:50.815003 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:50.815003 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:50.815003 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:50.815003 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:50.815003 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:50.815003 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:50.815003 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:03:50.815003 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:03:50.815003 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:03:50.815003 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:03:50.815980 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:50.815980 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:50.815980 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:50.815980 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:50.815980 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:50.815980 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:50.815980 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:50.815980 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:50.815980 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:50.815980 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:50.815980 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:50.815980 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:50.815980 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:50.815980 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:50.815980 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:50.815980 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:50.815980 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:03:50.816901 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:03:50.816901 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:50.816901 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:50.816901 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 409/834] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:03:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:50.817124 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3260c67-8c14-4b5c-91b7-a8ba8fa19b3c None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:50.838899 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eefd2dc-169b-4faf-9398-3657b5947405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:50.839529 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eefd2dc-169b-4faf-9398-3657b5947405 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:50.839812 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eefd2dc-169b-4faf-9398-3657b5947405 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:50.840089 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eefd2dc-169b-4faf-9398-3657b5947405 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:50.851642 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eefd2dc-169b-4faf-9398-3657b5947405 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:50.853176 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eefd2dc-169b-4faf-9398-3657b5947405 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:50.867533 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-6eefd2dc-169b-4faf-9398-3657b5947405 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:50.867533 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:50.867533 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:50.867533 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:50.867533 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:50.867533 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:50.867533 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:50.867533 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:50.867533 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:50.867533 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:50.867533 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:50.867533 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:50.867533 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:03:50.867533 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:03:50.867533 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:03:50.867533 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:03:50.868534 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:50.868534 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:50.868534 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:50.868534 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:50.868534 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:50.868534 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:50.868534 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:50.868534 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:50.868534 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:50.868534 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:50.868534 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:50.868534 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:50.868534 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:50.868534 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:50.868534 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:50.868534 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:50.868534 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:03:50.869465 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:03:50.869465 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:50.869465 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:50.873076 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 426/835] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:03:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:50.874055 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eefd2dc-169b-4faf-9398-3657b5947405 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:50.892193 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:50.925367 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3260c67-8c14-4b5c-91b7-a8ba8fa19b3c None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:50.958959 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a0a83a-4229-4955-acd4-b28c46ff5d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:50.959591 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a0a83a-4229-4955-acd4-b28c46ff5d19 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:50.959867 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a0a83a-4229-4955-acd4-b28c46ff5d19 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:50.960137 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a0a83a-4229-4955-acd4-b28c46ff5d19 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:50.960724 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a0a83a-4229-4955-acd4-b28c46ff5d19 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:50.962134 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a0a83a-4229-4955-acd4-b28c46ff5d19 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:50.970911 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-f2a0a83a-4229-4955-acd4-b28c46ff5d19 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 16:03:50.970911 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:50.970911 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 16:03:50.970911 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 16:03:50.970911 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 16:03:50.970911 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 16:03:50.970911 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 16:03:50.970911 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:03:50.970911 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 16:03:50.970911 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:50.970911 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 16:03:50.970911 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:50.970911 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:50.970911 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:50.970911 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:50.970911 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:50.970911 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:50.972064 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:50.972064 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:50.972064 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:50.972064 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:50.972064 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:50.972064 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:50.972064 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 16:03:50.972064 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 16:03:50.972064 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 16:03:50.972064 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 16:03:50.972064 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:50.972064 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:50.972064 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:50.972064 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:50.972064 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:50.972064 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:50.972953 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:50.972953 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:50.972953 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:50.972953 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:50.972953 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:50.972953 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:50.972953 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:50.972953 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:50.972953 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:50.972953 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:50.972953 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 16:03:50.972953 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:03:50.972953 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 16:03:50.972953 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:50.973698 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eefd2dc-169b-4faf-9398-3657b5947405 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:50.974106 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 410/836] 10.222.0.43 () {56 vars in 1114 bytes} [Tue Sep 6 16:03:50 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:50.974106 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3647075-aa8a-4ecd-9497-b32fa6d20f0f None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:50.974106 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3647075-aa8a-4ecd-9497-b32fa6d20f0f None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:50.974106 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3647075-aa8a-4ecd-9497-b32fa6d20f0f None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:50.976795 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 411/837] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:03:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:03:50.976795 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3647075-aa8a-4ecd-9497-b32fa6d20f0f None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:50.997665 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:51.008925 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ef3e8b-50ec-4f01-8620-17a0c78a7c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:51.009525 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ef3e8b-50ec-4f01-8620-17a0c78a7c27 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:51.009796 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ef3e8b-50ec-4f01-8620-17a0c78a7c27 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:51.010065 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ef3e8b-50ec-4f01-8620-17a0c78a7c27 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:51.010926 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ef3e8b-50ec-4f01-8620-17a0c78a7c27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:51.011292 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ef3e8b-50ec-4f01-8620-17a0c78a7c27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:51.012660 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ef3e8b-50ec-4f01-8620-17a0c78a7c27 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:51.023950 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 427/838] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:03:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:51.024721 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ef3e8b-50ec-4f01-8620-17a0c78a7c27 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:51.077863 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3647075-aa8a-4ecd-9497-b32fa6d20f0f None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:51.115227 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:51.120740 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20694c4-fd97-4789-a476-f116103cf0b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:51.121324 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20694c4-fd97-4789-a476-f116103cf0b9 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:51.121600 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20694c4-fd97-4789-a476-f116103cf0b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:51.121871 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20694c4-fd97-4789-a476-f116103cf0b9 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:51.122515 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20694c4-fd97-4789-a476-f116103cf0b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:51.122814 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20694c4-fd97-4789-a476-f116103cf0b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:51.123572 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ef3e8b-50ec-4f01-8620-17a0c78a7c27 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:51.124214 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20694c4-fd97-4789-a476-f116103cf0b9 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:51.136477 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 412/839] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:03:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:51.137906 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef94182-9c13-44c9-8599-66b482b75940 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:51.138180 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef94182-9c13-44c9-8599-66b482b75940 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:51.138442 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef94182-9c13-44c9-8599-66b482b75940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:51.155333 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-0ef94182-9c13-44c9-8599-66b482b75940 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:03:51.156511 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0ef94182-9c13-44c9-8599-66b482b75940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:03:51.156994 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323978ea-0d66-4a5c-b8a9-907adeaaf066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:51.157601 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323978ea-0d66-4a5c-b8a9-907adeaaf066 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:51.157874 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323978ea-0d66-4a5c-b8a9-907adeaaf066 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:51.158130 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323978ea-0d66-4a5c-b8a9-907adeaaf066 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:51.158786 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323978ea-0d66-4a5c-b8a9-907adeaaf066 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:51.159079 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323978ea-0d66-4a5c-b8a9-907adeaaf066 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:51.160434 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323978ea-0d66-4a5c-b8a9-907adeaaf066 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:51.164807 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 428/840] 10.222.0.43 () {56 vars in 1114 bytes} [Tue Sep 6 16:03:51 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:51.165436 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323978ea-0d66-4a5c-b8a9-907adeaaf066 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:51.213127 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:51.226447 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 413/841] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:03:51 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:03:51.227301 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef94182-9c13-44c9-8599-66b482b75940 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:51.266255 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323978ea-0d66-4a5c-b8a9-907adeaaf066 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:51.303469 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962a5251-f4a4-4e52-ab0e-5971f7e124c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:51.304608 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a5251-f4a4-4e52-ab0e-5971f7e124c0 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:51.304608 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a5251-f4a4-4e52-ab0e-5971f7e124c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:51.304608 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a5251-f4a4-4e52-ab0e-5971f7e124c0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:51.313595 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962a5251-f4a4-4e52-ab0e-5971f7e124c0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:51.315148 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962a5251-f4a4-4e52-ab0e-5971f7e124c0 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:51.326457 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-962a5251-f4a4-4e52-ab0e-5971f7e124c0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:03:51.326457 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:51.326457 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:51.326457 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:51.326457 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:51.326457 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:51.326457 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:51.326457 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:51.326457 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:51.326457 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:51.326457 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:51.326457 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:51.326457 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:03:51.326457 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:03:51.326457 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:03:51.327350 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:03:51.327350 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:51.327350 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:51.327350 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:51.327350 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:51.327350 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:51.327350 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:51.327350 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:51.327350 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:51.327350 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:51.327350 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:51.327350 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:51.327350 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:51.327350 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:51.327350 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:51.327350 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:51.327350 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:51.328213 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:03:51.328213 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:03:51.328213 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:03:51.328213 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:03:51.328213 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:03:51.328213 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:03:51.328213 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:03:51.328213 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:51.328213 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 429/842] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 6 16:03:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:51.328688 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962a5251-f4a4-4e52-ab0e-5971f7e124c0 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:51.348734 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef94182-9c13-44c9-8599-66b482b75940 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:51.372229 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:51.386277 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d175cb3-fefc-4434-897b-9a200c686af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:51.387227 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d175cb3-fefc-4434-897b-9a200c686af3 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:51.387514 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d175cb3-fefc-4434-897b-9a200c686af3 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:51.387919 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d175cb3-fefc-4434-897b-9a200c686af3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:51.395081 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d175cb3-fefc-4434-897b-9a200c686af3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:51.396533 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d175cb3-fefc-4434-897b-9a200c686af3 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:51.407920 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-4d175cb3-fefc-4434-897b-9a200c686af3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 16:03:51.407920 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:51.407920 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:51.407920 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:51.407920 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:51.407920 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:51.407920 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:51.407920 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:51.407920 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:51.407920 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:51.407920 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:51.407920 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:51.407920 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:03:51.407920 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:03:51.407920 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:03:51.407920 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:03:51.408904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:51.408904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:51.408904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:03:51.408904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:03:51.408904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:03:51.408904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:03:51.408904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:51.408904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:51.408904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:51.408904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:51.408904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:51.408904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:51.408904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:51.408904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:51.408904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:51.408904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:51.408904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:51.408904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:51.409853 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:51.409853 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:51.409853 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:03:51.409853 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:03:51.409853 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:03:51.409853 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:03:51.409853 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:03:51.409853 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:03:51.409853 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 16:03:51.409853 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:51.413942 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 414/843] 10.222.0.43 () {56 vars in 1117 bytes} [Tue Sep 6 16:03:51 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:51.414703 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d175cb3-fefc-4434-897b-9a200c686af3 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:51.434664 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962a5251-f4a4-4e52-ab0e-5971f7e124c0 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:51.444405 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:51.472273 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74916f38-0e1f-49f9-8773-e95369c91828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:51.472911 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74916f38-0e1f-49f9-8773-e95369c91828 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:51.473213 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74916f38-0e1f-49f9-8773-e95369c91828 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:51.473450 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74916f38-0e1f-49f9-8773-e95369c91828 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:51.474075 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74916f38-0e1f-49f9-8773-e95369c91828 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:51.474372 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74916f38-0e1f-49f9-8773-e95369c91828 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:51.475810 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74916f38-0e1f-49f9-8773-e95369c91828 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:51.480324 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 430/844] 10.222.0.43 () {56 vars in 1114 bytes} [Tue Sep 6 16:03:51 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:51.481128 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74916f38-0e1f-49f9-8773-e95369c91828 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:51.532208 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d175cb3-fefc-4434-897b-9a200c686af3 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:51.575914 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101909fc-a5ad-4a6d-9413-2a83bf3a3ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:51.576537 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101909fc-a5ad-4a6d-9413-2a83bf3a3ea9 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:51.576964 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101909fc-a5ad-4a6d-9413-2a83bf3a3ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:51.577241 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101909fc-a5ad-4a6d-9413-2a83bf3a3ea9 None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:51.577962 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101909fc-a5ad-4a6d-9413-2a83bf3a3ea9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:51.578249 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101909fc-a5ad-4a6d-9413-2a83bf3a3ea9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:51.580076 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101909fc-a5ad-4a6d-9413-2a83bf3a3ea9 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:51.599724 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74916f38-0e1f-49f9-8773-e95369c91828 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:51.602485 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 415/845] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 6 16:03:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:51.602485 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101909fc-a5ad-4a6d-9413-2a83bf3a3ea9 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:51.644325 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e9a280-2404-4dc0-8818-8b6ae0c0a80e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:51.645539 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e9a280-2404-4dc0-8818-8b6ae0c0a80e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:51.645539 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e9a280-2404-4dc0-8818-8b6ae0c0a80e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:51.645539 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e9a280-2404-4dc0-8818-8b6ae0c0a80e None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:51.646130 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e9a280-2404-4dc0-8818-8b6ae0c0a80e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:51.646427 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e9a280-2404-4dc0-8818-8b6ae0c0a80e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:51.647859 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e9a280-2404-4dc0-8818-8b6ae0c0a80e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:51.652026 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:51.671317 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 431/846] 10.222.0.43 () {56 vars in 1149 bytes} [Tue Sep 6 16:03:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:51.671317 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e9a280-2404-4dc0-8818-8b6ae0c0a80e None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:51.700031 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:51.722864 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101909fc-a5ad-4a6d-9413-2a83bf3a3ea9 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:51.760350 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb578ad-de58-4474-be22-9e2404576c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:51.761017 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb578ad-de58-4474-be22-9e2404576c84 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:51.761293 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb578ad-de58-4474-be22-9e2404576c84 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:51.761560 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb578ad-de58-4474-be22-9e2404576c84 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:51.770309 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e9a280-2404-4dc0-8818-8b6ae0c0a80e None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:51.772014 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb578ad-de58-4474-be22-9e2404576c84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:03:51.776811 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-ebb578ad-de58-4474-be22-9e2404576c84 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:03:51.776811 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:51.776811 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:51.776811 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:51.776811 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:51.776811 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:51.776811 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:51.776811 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:51.776811 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:51.776811 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:51.776811 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:51.776811 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:51.776811 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:03:51.776811 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:03:51.776811 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:03:51.776811 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:03:51.777734 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:03:51.777734 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:03:51.777734 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:03:51.777734 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:03:51.777734 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:51.777734 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:51.777734 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:51.777734 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:51.777734 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:51.777734 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:51.777734 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:51.777734 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:51.777734 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:51.777734 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:51.777734 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:51.777734 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:51.781686 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:51.781686 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:51.781686 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:51.781686 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:51.781686 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:03:51.781686 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:03:51.781686 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:03:51.781686 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:03:51.781686 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:03:51.781686 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:03:51.781686 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:03:51.781686 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:51.781686 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 416/847] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 6 16:03:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:51.781686 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb578ad-de58-4474-be22-9e2404576c84 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:51.810427 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5dab5c-eb96-49dc-a421-5c8ec79395bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:51.811032 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5dab5c-eb96-49dc-a421-5c8ec79395bf None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:51.811306 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5dab5c-eb96-49dc-a421-5c8ec79395bf None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:51.811572 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5dab5c-eb96-49dc-a421-5c8ec79395bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:51.812283 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5dab5c-eb96-49dc-a421-5c8ec79395bf None admin] RBAC: Adding query filter params (role.id=edaabdea8be84a7980fb2c651fe52f63, scope.project.id=128d210e8dad48ac921ed4cb1d444ceb, user.id=63de41f4348243998029f7d07aa5dc40) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:51.812594 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5dab5c-eb96-49dc-a421-5c8ec79395bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:51.814232 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5dab5c-eb96-49dc-a421-5c8ec79395bf None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:51.822656 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 432/848] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 6 16:03:51 2022] GET /identity/v3/role_assignments?user.id=63de41f4348243998029f7d07aa5dc40&scope.project.id=128d210e8dad48ac921ed4cb1d444ceb&role.id=edaabdea8be84a7980fb2c651fe52f63 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:51.823693 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5dab5c-eb96-49dc-a421-5c8ec79395bf None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:51.894770 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb578ad-de58-4474-be22-9e2404576c84 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:51.897891 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:51.933777 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:51.943598 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5dab5c-eb96-49dc-a421-5c8ec79395bf None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:51.992536 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526b8c83-17eb-44df-a3d1-f9d2cba4c2c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:51.993914 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526b8c83-17eb-44df-a3d1-f9d2cba4c2c0 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:51.994347 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526b8c83-17eb-44df-a3d1-f9d2cba4c2c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:51.994800 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526b8c83-17eb-44df-a3d1-f9d2cba4c2c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:51.995632 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526b8c83-17eb-44df-a3d1-f9d2cba4c2c0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:51.996701 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526b8c83-17eb-44df-a3d1-f9d2cba4c2c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:51.998427 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526b8c83-17eb-44df-a3d1-f9d2cba4c2c0 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:52.010247 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 433/849] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 6 16:03:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:52.015841 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526b8c83-17eb-44df-a3d1-f9d2cba4c2c0 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:52.065445 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a51eac2-3404-43c4-880e-47768c4faea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:52.066169 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a51eac2-3404-43c4-880e-47768c4faea2 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:52.066444 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a51eac2-3404-43c4-880e-47768c4faea2 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:52.066715 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a51eac2-3404-43c4-880e-47768c4faea2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:52.139469 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a51eac2-3404-43c4-880e-47768c4faea2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:52.141219 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a51eac2-3404-43c4-880e-47768c4faea2 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:52.143479 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526b8c83-17eb-44df-a3d1-f9d2cba4c2c0 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:52.197982 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1e404e-5651-4033-9861-b02e680c8504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:52.199465 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1e404e-5651-4033-9861-b02e680c8504 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:52.200210 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1e404e-5651-4033-9861-b02e680c8504 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:52.200668 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1e404e-5651-4033-9861-b02e680c8504 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:52.215076 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1e404e-5651-4033-9861-b02e680c8504 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:52.216632 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1e404e-5651-4033-9861-b02e680c8504 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:52.235263 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-2e1e404e-5651-4033-9861-b02e680c8504 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:52.235263 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:52.235263 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:52.235263 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:52.235263 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:52.235263 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:52.235263 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:52.235263 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:52.235263 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:52.235263 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:52.235263 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:52.235263 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:52.235263 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:03:52.235263 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:03:52.235263 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:03:52.235263 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:03:52.236355 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:52.236355 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:52.236355 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:52.236355 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:52.236355 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:52.236355 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:52.236355 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:52.236355 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:52.236355 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:52.236355 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:52.236355 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:52.236355 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:52.236355 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:52.236355 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:52.236355 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:52.236355 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:52.236355 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:03:52.237292 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:03:52.237292 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:52.237292 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:52.237292 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 434/850] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:03:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:03:52.244937 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1e404e-5651-4033-9861-b02e680c8504 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:52.257567 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:52.258199 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 417/851] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 6 16:03:51 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 16:03:52.264376 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a51eac2-3404-43c4-880e-47768c4faea2 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:52.266623 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:52.353688 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1e404e-5651-4033-9861-b02e680c8504 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:52.372980 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a51eac2-3404-43c4-880e-47768c4faea2 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:52.388222 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484cbe77-c8f2-4553-8dfa-3a355b62a81d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:52.389056 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484cbe77-c8f2-4553-8dfa-3a355b62a81d None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:52.389271 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484cbe77-c8f2-4553-8dfa-3a355b62a81d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:52.389534 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484cbe77-c8f2-4553-8dfa-3a355b62a81d None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:52.390175 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484cbe77-c8f2-4553-8dfa-3a355b62a81d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:52.390473 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484cbe77-c8f2-4553-8dfa-3a355b62a81d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:52.391992 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484cbe77-c8f2-4553-8dfa-3a355b62a81d None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:52.408901 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 435/852] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:03:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:03:52.410485 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484cbe77-c8f2-4553-8dfa-3a355b62a81d None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:52.413746 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65db571f-741c-495e-9b43-e3b1c2f888a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:52.415374 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65db571f-741c-495e-9b43-e3b1c2f888a9 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:52.415374 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65db571f-741c-495e-9b43-e3b1c2f888a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:52.415374 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65db571f-741c-495e-9b43-e3b1c2f888a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:52.425874 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65db571f-741c-495e-9b43-e3b1c2f888a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:52.427331 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65db571f-741c-495e-9b43-e3b1c2f888a9 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:52.442497 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-65db571f-741c-495e-9b43-e3b1c2f888a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:52.442497 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:52.442497 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:52.442497 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:52.442497 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:52.442497 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:52.442497 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:52.442497 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:52.442497 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:52.442497 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:52.442497 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:52.442497 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:52.442497 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:03:52.442497 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:03:52.442497 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:03:52.442497 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:03:52.443470 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:52.443470 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:52.443470 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:52.443470 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:52.443470 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:52.443470 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:52.443470 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:52.443470 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:52.443470 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:52.443470 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:52.443470 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:52.443470 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:52.443470 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:52.443470 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:52.443470 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:52.443470 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:52.443470 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:03:52.444355 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:03:52.444355 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:52.444355 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:52.448357 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 418/853] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:03:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:03:52.449186 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65db571f-741c-495e-9b43-e3b1c2f888a9 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:52.484171 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:52.509431 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:52.526836 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484cbe77-c8f2-4553-8dfa-3a355b62a81d None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:52.565535 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d413ce-ca61-4e1d-a8ab-ef663992ccdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:52.566578 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d413ce-ca61-4e1d-a8ab-ef663992ccdf None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:52.567046 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d413ce-ca61-4e1d-a8ab-ef663992ccdf None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:52.567532 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d413ce-ca61-4e1d-a8ab-ef663992ccdf None admin] PATH_INFO: `/v3/users/swift` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:52.568622 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65db571f-741c-495e-9b43-e3b1c2f888a9 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:52.575713 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d413ce-ca61-4e1d-a8ab-ef663992ccdf None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:52.577657 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d413ce-ca61-4e1d-a8ab-ef663992ccdf None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:52.591438 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-88d413ce-ca61-4e1d-a8ab-ef663992ccdf None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 06 16:03:52.591438 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:52.591438 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:52.591438 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:52.591438 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:52.591438 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:52.591438 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:52.591438 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:52.591438 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:52.591438 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:52.591438 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:52.591438 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:52.591438 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:03:52.591438 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:03:52.591438 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:03:52.591438 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:03:52.592453 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:52.592453 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:52.592453 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:03:52.592453 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:03:52.592453 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:03:52.592453 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:03:52.592453 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:52.592453 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:52.592453 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:52.592453 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:52.592453 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:52.592453 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:52.592453 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:52.592453 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:52.592453 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:52.592453 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:52.592453 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:52.592453 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:52.593463 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:52.593463 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:52.593463 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:03:52.593463 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:03:52.593463 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:03:52.593463 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:03:52.593463 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:03:52.593463 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:03:52.593463 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 06 16:03:52.593463 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:52.601077 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 436/854] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 6 16:03:52 2022] GET /identity/v3/users/swift => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:52.603011 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d413ce-ca61-4e1d-a8ab-ef663992ccdf None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:52.609069 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82548775-d6b5-446d-b6be-78fe3a06d3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:52.609687 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82548775-d6b5-446d-b6be-78fe3a06d3c1 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:52.609966 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82548775-d6b5-446d-b6be-78fe3a06d3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:52.610226 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82548775-d6b5-446d-b6be-78fe3a06d3c1 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:52.610875 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82548775-d6b5-446d-b6be-78fe3a06d3c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:52.611171 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82548775-d6b5-446d-b6be-78fe3a06d3c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:52.612649 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82548775-d6b5-446d-b6be-78fe3a06d3c1 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:52.627540 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 419/855] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:03:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:52.633446 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82548775-d6b5-446d-b6be-78fe3a06d3c1 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:52.718710 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d413ce-ca61-4e1d-a8ab-ef663992ccdf None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:52.746481 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:52.756650 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791186f8-9de4-4b7c-98f8-b6e69713d5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:52.757666 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791186f8-9de4-4b7c-98f8-b6e69713d5dc None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:52.758219 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791186f8-9de4-4b7c-98f8-b6e69713d5dc None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:52.758678 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791186f8-9de4-4b7c-98f8-b6e69713d5dc None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:52.759613 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791186f8-9de4-4b7c-98f8-b6e69713d5dc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:52.760211 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791186f8-9de4-4b7c-98f8-b6e69713d5dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:52.761949 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791186f8-9de4-4b7c-98f8-b6e69713d5dc None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:52.763846 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82548775-d6b5-446d-b6be-78fe3a06d3c1 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:52.780737 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:52.784784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 437/856] 10.222.0.43 () {56 vars in 1135 bytes} [Tue Sep 6 16:03:52 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:03:52.787840 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791186f8-9de4-4b7c-98f8-b6e69713d5dc None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:52.806619 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c76a89-4cd6-4781-8cb2-b893d8c95a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:52.807261 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c76a89-4cd6-4781-8cb2-b893d8c95a58 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:52.807535 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c76a89-4cd6-4781-8cb2-b893d8c95a58 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:52.807798 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c76a89-4cd6-4781-8cb2-b893d8c95a58 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:52.816081 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c76a89-4cd6-4781-8cb2-b893d8c95a58 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:52.818202 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c76a89-4cd6-4781-8cb2-b893d8c95a58 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:52.836394 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-45c76a89-4cd6-4781-8cb2-b893d8c95a58 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:03:52.836394 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:52.836394 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:52.836394 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:52.836394 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:52.836394 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:52.836394 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:52.836394 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:52.836394 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:52.836394 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:52.836394 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:52.836394 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:52.836394 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:03:52.836394 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:03:52.836394 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:03:52.837381 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:03:52.837381 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:52.837381 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:52.837381 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:52.837381 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:52.837381 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:52.837381 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:52.837381 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:52.837381 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:52.837381 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:52.837381 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:52.837381 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:52.837381 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:52.837381 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:52.837381 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:52.837381 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:52.837381 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:52.838276 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:03:52.838276 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:03:52.838276 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:03:52.838276 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:03:52.838276 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:03:52.838276 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:03:52.838276 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:03:52.838276 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:52.840823 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 420/857] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 6 16:03:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:52.843116 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c76a89-4cd6-4781-8cb2-b893d8c95a58 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:52.915391 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791186f8-9de4-4b7c-98f8-b6e69713d5dc None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:52.943804 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c76a89-4cd6-4781-8cb2-b893d8c95a58 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:52.951935 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d444561b-c55f-4464-a627-64193c1a94e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:52.952535 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d444561b-c55f-4464-a627-64193c1a94e0 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:52.952833 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d444561b-c55f-4464-a627-64193c1a94e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:52.953104 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d444561b-c55f-4464-a627-64193c1a94e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:52.978110 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56b4bb7-68b2-4555-be73-9fac8f3037ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:52.978641 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b4bb7-68b2-4555-be73-9fac8f3037ef None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:52.978919 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b4bb7-68b2-4555-be73-9fac8f3037ef None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:52.979191 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b4bb7-68b2-4555-be73-9fac8f3037ef None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:52.979895 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56b4bb7-68b2-4555-be73-9fac8f3037ef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:52.980190 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56b4bb7-68b2-4555-be73-9fac8f3037ef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:52.981685 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56b4bb7-68b2-4555-be73-9fac8f3037ef None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:52.994399 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 421/858] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 6 16:03:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:03:53.000908 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56b4bb7-68b2-4555-be73-9fac8f3037ef None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:53.015020 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:53.031387 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d444561b-c55f-4464-a627-64193c1a94e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:53.032788 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d444561b-c55f-4464-a627-64193c1a94e0 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:53.057134 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:53.114104 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56b4bb7-68b2-4555-be73-9fac8f3037ef None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:53.134499 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 438/859] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 6 16:03:52 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 06 16:03:53.139791 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d444561b-c55f-4464-a627-64193c1a94e0 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:53.150757 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a74160c-1fde-4f68-a435-dee231380ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:53.151645 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a74160c-1fde-4f68-a435-dee231380ea2 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:53.152158 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a74160c-1fde-4f68-a435-dee231380ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:53.152579 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a74160c-1fde-4f68-a435-dee231380ea2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:53.153575 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a74160c-1fde-4f68-a435-dee231380ea2 None admin] RBAC: Adding query filter params (role.id=31321627b02c4ff188f8fe1521ce3181, scope.project.id=128d210e8dad48ac921ed4cb1d444ceb, user.id=d865006413b24ec3922619311f81f353) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:53.154021 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a74160c-1fde-4f68-a435-dee231380ea2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:53.155705 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a74160c-1fde-4f68-a435-dee231380ea2 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:53.162561 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 422/860] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 6 16:03:52 2022] GET /identity/v3/role_assignments?user.id=d865006413b24ec3922619311f81f353&scope.project.id=128d210e8dad48ac921ed4cb1d444ceb&role.id=31321627b02c4ff188f8fe1521ce3181 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:53.235854 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:53.259442 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d444561b-c55f-4464-a627-64193c1a94e0 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:53.301668 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a37553-180d-4a65-af28-92ffa72e83c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:53.302319 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a37553-180d-4a65-af28-92ffa72e83c5 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:53.302593 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a37553-180d-4a65-af28-92ffa72e83c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:53.302867 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a37553-180d-4a65-af28-92ffa72e83c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:53.314113 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a37553-180d-4a65-af28-92ffa72e83c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:53.316049 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a37553-180d-4a65-af28-92ffa72e83c5 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:53.337608 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-14a37553-180d-4a65-af28-92ffa72e83c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:53.337608 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:53.337608 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:53.337608 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:53.337608 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:53.337608 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:53.337608 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:53.337608 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:53.337608 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:53.337608 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:53.337608 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:53.337608 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:53.337608 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:03:53.337608 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:03:53.337608 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:03:53.337608 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:03:53.338787 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:53.338787 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:53.338787 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:53.338787 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:53.338787 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:53.338787 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:53.338787 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:53.338787 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:53.338787 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:53.338787 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:53.338787 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:53.338787 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:53.338787 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:53.338787 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:53.338787 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:53.338787 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:53.338787 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:03:53.340414 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:03:53.340414 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:53.340414 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:53.340414 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 439/861] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:03:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:53.344909 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a74160c-1fde-4f68-a435-dee231380ea2 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:53.426286 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341ac0a7-8ff6-4926-8b2c-e1e5f6f1e1ad None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:53.426716 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341ac0a7-8ff6-4926-8b2c-e1e5f6f1e1ad None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:53.427081 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341ac0a7-8ff6-4926-8b2c-e1e5f6f1e1ad None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:53.428663 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 440/862] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:03:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 16:03:53.434069 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d660042-82a2-42a3-845f-faf6a03a9b02 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:53.434566 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d660042-82a2-42a3-845f-faf6a03a9b02 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:53.434988 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d660042-82a2-42a3-845f-faf6a03a9b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:53.454493 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:53.456472 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-4d660042-82a2-42a3-845f-faf6a03a9b02 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:03:53.458214 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-4d660042-82a2-42a3-845f-faf6a03a9b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:03:53.460413 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a74160c-1fde-4f68-a435-dee231380ea2 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:53.498321 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2acfca-a7c8-4ca0-878d-60718e3171b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:53.499166 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2acfca-a7c8-4ca0-878d-60718e3171b4 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:53.499585 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2acfca-a7c8-4ca0-878d-60718e3171b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:53.499992 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2acfca-a7c8-4ca0-878d-60718e3171b4 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:53.500841 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2acfca-a7c8-4ca0-878d-60718e3171b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:53.501371 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2acfca-a7c8-4ca0-878d-60718e3171b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:53.503042 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2acfca-a7c8-4ca0-878d-60718e3171b4 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:53.515857 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 423/863] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:03:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:53.521364 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2acfca-a7c8-4ca0-878d-60718e3171b4 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:53.547890 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 441/864] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:03:53 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:03:53.631178 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2acfca-a7c8-4ca0-878d-60718e3171b4 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:53.666864 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197e197c-b5e5-40f1-bfe0-6df7575a69e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:53.667546 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197e197c-b5e5-40f1-bfe0-6df7575a69e3 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:53.667818 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197e197c-b5e5-40f1-bfe0-6df7575a69e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:53.668133 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197e197c-b5e5-40f1-bfe0-6df7575a69e3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:53.676377 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197e197c-b5e5-40f1-bfe0-6df7575a69e3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:53.678622 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197e197c-b5e5-40f1-bfe0-6df7575a69e3 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:53.692588 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-197e197c-b5e5-40f1-bfe0-6df7575a69e3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:03:53.692588 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:53.692588 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:53.692588 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:53.692588 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:53.692588 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:53.692588 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:53.692588 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:53.692588 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:53.692588 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:53.692588 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:53.692588 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:53.692588 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:03:53.692588 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:03:53.692588 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:03:53.693562 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:03:53.693562 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:53.693562 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:53.693562 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:53.693562 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:53.693562 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:53.693562 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:53.693562 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:53.693562 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:53.693562 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:53.693562 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:53.693562 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:53.693562 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:53.693562 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:53.693562 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:53.693562 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:53.693562 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:53.694436 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:03:53.694436 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:03:53.694436 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:03:53.694436 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:03:53.694436 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:03:53.694436 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:03:53.694436 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:03:53.694436 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:53.694436 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 424/865] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 6 16:03:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:53.701962 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d660042-82a2-42a3-845f-faf6a03a9b02 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:53.769171 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:53.803198 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d660042-82a2-42a3-845f-faf6a03a9b02 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:53.814109 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197e197c-b5e5-40f1-bfe0-6df7575a69e3 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:53.834777 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:53.843869 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e93fc0e-99b4-433a-8e29-1c546ced794c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:53.844741 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e93fc0e-99b4-433a-8e29-1c546ced794c None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:53.845043 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e93fc0e-99b4-433a-8e29-1c546ced794c None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:53.845313 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e93fc0e-99b4-433a-8e29-1c546ced794c None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:53.846047 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e93fc0e-99b4-433a-8e29-1c546ced794c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:53.846496 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e93fc0e-99b4-433a-8e29-1c546ced794c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:53.848583 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e93fc0e-99b4-433a-8e29-1c546ced794c None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:53.864010 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 442/866] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 6 16:03:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:53.869944 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e93fc0e-99b4-433a-8e29-1c546ced794c None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:53.924239 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197e197c-b5e5-40f1-bfe0-6df7575a69e3 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:53.963682 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ea6737-2d37-4746-af4c-5ea7bdc487ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:53.964296 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea6737-2d37-4746-af4c-5ea7bdc487ce None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:53.964570 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea6737-2d37-4746-af4c-5ea7bdc487ce None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:53.964860 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea6737-2d37-4746-af4c-5ea7bdc487ce None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:53.965554 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ea6737-2d37-4746-af4c-5ea7bdc487ce None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:53.967008 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ea6737-2d37-4746-af4c-5ea7bdc487ce None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:53.976130 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e93fc0e-99b4-433a-8e29-1c546ced794c None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:53.987459 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 425/867] 10.222.0.43 () {60 vars in 1141 bytes} [Tue Sep 6 16:03:53 2022] POST /identity/v3/services => generated 251 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:03:53.988663 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5b1ea7-e990-4104-b6cc-bcd5e6ba72ab None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:53.988989 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5b1ea7-e990-4104-b6cc-bcd5e6ba72ab None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:53.989260 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5b1ea7-e990-4104-b6cc-bcd5e6ba72ab None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:53.990617 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 426/868] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:03:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:03:53.996236 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d06cfe5-3e69-47f7-9c45-c1c4a2f74c5a None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:53.996495 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d06cfe5-3e69-47f7-9c45-c1c4a2f74c5a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:53.996790 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d06cfe5-3e69-47f7-9c45-c1c4a2f74c5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:54.017817 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-0d06cfe5-3e69-47f7-9c45-c1c4a2f74c5a None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:03:54.018937 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0d06cfe5-3e69-47f7-9c45-c1c4a2f74c5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:03:54.024414 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:54.047366 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1090ef2-c232-4244-8b59-9fd7675ab1e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:54.048693 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1090ef2-c232-4244-8b59-9fd7675ab1e1 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:54.049516 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1090ef2-c232-4244-8b59-9fd7675ab1e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:54.049996 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1090ef2-c232-4244-8b59-9fd7675ab1e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:54.051624 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1090ef2-c232-4244-8b59-9fd7675ab1e1 None admin] RBAC: Adding query filter params (role.id=edaabdea8be84a7980fb2c651fe52f63, scope.project.id=128d210e8dad48ac921ed4cb1d444ceb, user.id=312026c080b64aff9dd483d3591288a6) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:54.052209 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1090ef2-c232-4244-8b59-9fd7675ab1e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:54.054989 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1090ef2-c232-4244-8b59-9fd7675ab1e1 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:54.065516 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 443/869] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 6 16:03:53 2022] GET /identity/v3/role_assignments?user.id=312026c080b64aff9dd483d3591288a6&scope.project.id=128d210e8dad48ac921ed4cb1d444ceb&role.id=edaabdea8be84a7980fb2c651fe52f63 => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:03:54.114073 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 427/870] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:03:53 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:03:54.389822 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1090ef2-c232-4244-8b59-9fd7675ab1e1 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:54.494643 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1090ef2-c232-4244-8b59-9fd7675ab1e1 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:54.529190 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677822eb-541b-4753-ba7f-1b689cbcefb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:54.529753 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677822eb-541b-4753-ba7f-1b689cbcefb8 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:54.530031 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677822eb-541b-4753-ba7f-1b689cbcefb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:54.530306 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677822eb-541b-4753-ba7f-1b689cbcefb8 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:54.530892 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677822eb-541b-4753-ba7f-1b689cbcefb8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:54.532260 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677822eb-541b-4753-ba7f-1b689cbcefb8 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:54.541474 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-677822eb-541b-4753-ba7f-1b689cbcefb8 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 16:03:54.541474 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:54.541474 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 16:03:54.541474 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 16:03:54.541474 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 16:03:54.541474 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 16:03:54.541474 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 16:03:54.541474 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:03:54.541474 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 16:03:54.541474 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:54.541474 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 16:03:54.541474 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:54.541474 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:54.541474 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:54.541474 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:54.541474 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:54.542425 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:54.542425 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:54.542425 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:54.542425 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:54.542425 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:54.542425 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:54.542425 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:54.542425 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 16:03:54.542425 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 16:03:54.542425 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 16:03:54.542425 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 16:03:54.542425 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:54.542425 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:54.542425 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:54.542425 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:54.542425 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:54.542425 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:54.543769 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:54.543769 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:54.543769 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:54.543769 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:54.543769 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:54.543769 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:54.543769 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:54.543769 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:54.543769 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:54.543769 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:54.543769 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 16:03:54.543769 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:03:54.543769 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 16:03:54.543769 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:54.543769 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 444/871] 10.222.0.43 () {56 vars in 1132 bytes} [Tue Sep 6 16:03:54 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:54.548643 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d06cfe5-3e69-47f7-9c45-c1c4a2f74c5a None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:54.559597 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:54.655508 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d06cfe5-3e69-47f7-9c45-c1c4a2f74c5a None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:54.695273 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea895b1-630c-43c9-80af-ca852223ff88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:54.696194 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea895b1-630c-43c9-80af-ca852223ff88 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:54.696685 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea895b1-630c-43c9-80af-ca852223ff88 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:54.697150 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea895b1-630c-43c9-80af-ca852223ff88 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:54.697957 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea895b1-630c-43c9-80af-ca852223ff88 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:54.698456 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea895b1-630c-43c9-80af-ca852223ff88 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:54.700039 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea895b1-630c-43c9-80af-ca852223ff88 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:54.705793 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 428/872] 10.222.0.43 () {56 vars in 1126 bytes} [Tue Sep 6 16:03:54 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:54.711367 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677822eb-541b-4753-ba7f-1b689cbcefb8 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:54.769934 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:54.817191 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677822eb-541b-4753-ba7f-1b689cbcefb8 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:54.851466 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39b7940-f22b-40a8-8415-6aa90e92ce64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:54.852077 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39b7940-f22b-40a8-8415-6aa90e92ce64 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:54.852358 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39b7940-f22b-40a8-8415-6aa90e92ce64 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:54.852640 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39b7940-f22b-40a8-8415-6aa90e92ce64 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:54.853302 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39b7940-f22b-40a8-8415-6aa90e92ce64 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:54.853604 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39b7940-f22b-40a8-8415-6aa90e92ce64 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:54.854976 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39b7940-f22b-40a8-8415-6aa90e92ce64 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:54.859802 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 445/873] 10.222.0.43 () {56 vars in 1126 bytes} [Tue Sep 6 16:03:54 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:55.278643 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e11ede-eff2-46b7-a000-8760f88a7498 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:55.279210 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e11ede-eff2-46b7-a000-8760f88a7498 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:55.279677 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e11ede-eff2-46b7-a000-8760f88a7498 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:55.281568 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 429/874] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:03:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:03:55.286853 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3568b117-e1e3-480b-af74-8bf6e1544310 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:55.287302 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3568b117-e1e3-480b-af74-8bf6e1544310 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:55.287398 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3568b117-e1e3-480b-af74-8bf6e1544310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:55.306231 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-3568b117-e1e3-480b-af74-8bf6e1544310 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:03:55.307415 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-3568b117-e1e3-480b-af74-8bf6e1544310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:03:55.385018 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 446/875] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:03:55 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:03:55.649913 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e11ede-eff2-46b7-a000-8760f88a7498 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:55.728110 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:55.760400 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e11ede-eff2-46b7-a000-8760f88a7498 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:55.794618 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c384e269-d22d-465d-b543-da01634d4a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:55.795488 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c384e269-d22d-465d-b543-da01634d4a1b None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:55.795909 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c384e269-d22d-465d-b543-da01634d4a1b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:55.796350 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c384e269-d22d-465d-b543-da01634d4a1b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:55.806749 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c384e269-d22d-465d-b543-da01634d4a1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:03:55.811771 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-c384e269-d22d-465d-b543-da01634d4a1b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 16:03:55.811771 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:55.811771 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:55.811771 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:55.811771 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:55.811771 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:55.811771 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:55.811771 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:55.811771 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:55.811771 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:55.811771 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:55.811771 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:55.811771 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:03:55.811771 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:03:55.811771 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:03:55.811771 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:03:55.812956 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:03:55.812956 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:03:55.812956 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:03:55.812956 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:03:55.812956 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:55.812956 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:55.812956 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:55.812956 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:55.812956 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:55.812956 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:55.812956 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:55.812956 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:55.812956 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:55.812956 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:55.812956 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:55.812956 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:55.813997 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:55.813997 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:55.813997 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:55.813997 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:55.813997 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:03:55.813997 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:03:55.813997 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:03:55.813997 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:03:55.813997 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:03:55.813997 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:03:55.813997 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 16:03:55.813997 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:55.816378 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 430/876] 10.222.0.43 () {56 vars in 1120 bytes} [Tue Sep 6 16:03:55 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:55.822032 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3568b117-e1e3-480b-af74-8bf6e1544310 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:55.868993 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:55.925222 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3568b117-e1e3-480b-af74-8bf6e1544310 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:55.964220 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c848f516-a7c5-4fb2-b6d0-dd4c4858942e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:55.964867 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c848f516-a7c5-4fb2-b6d0-dd4c4858942e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:55.965150 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c848f516-a7c5-4fb2-b6d0-dd4c4858942e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:55.965422 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c848f516-a7c5-4fb2-b6d0-dd4c4858942e None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:55.966094 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c848f516-a7c5-4fb2-b6d0-dd4c4858942e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:55.966394 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c848f516-a7c5-4fb2-b6d0-dd4c4858942e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:55.967769 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c848f516-a7c5-4fb2-b6d0-dd4c4858942e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:55.977142 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 447/877] 10.222.0.43 () {56 vars in 1115 bytes} [Tue Sep 6 16:03:55 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:55.983220 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c384e269-d22d-465d-b543-da01634d4a1b None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:55.997499 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5466861a-b383-4b34-9d82-4e56f4f04947 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:55.997823 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5466861a-b383-4b34-9d82-4e56f4f04947 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:55.998156 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5466861a-b383-4b34-9d82-4e56f4f04947 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:56.000799 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 448/878] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:03:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 16:03:56.004372 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011f0ab9-ae06-4cba-83d1-2ca974f93f8b None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:56.004654 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011f0ab9-ae06-4cba-83d1-2ca974f93f8b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:56.004973 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011f0ab9-ae06-4cba-83d1-2ca974f93f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:56.026387 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-011f0ab9-ae06-4cba-83d1-2ca974f93f8b None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:03:56.027605 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-011f0ab9-ae06-4cba-83d1-2ca974f93f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:03:56.099274 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 449/879] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:03:56 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 16:03:56.100842 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8877a9c0-d778-48cc-994c-ae9ad5e33263 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:56.101327 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8877a9c0-d778-48cc-994c-ae9ad5e33263 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:56.101787 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8877a9c0-d778-48cc-994c-ae9ad5e33263 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:56.103431 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 450/880] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:03:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:03:56.111858 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c384e269-d22d-465d-b543-da01634d4a1b None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:56.111990 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b604d1-8220-4553-aa44-4d1f92f549e1 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:56.111990 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b604d1-8220-4553-aa44-4d1f92f549e1 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:56.111990 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b604d1-8220-4553-aa44-4d1f92f549e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:56.131935 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-71b604d1-8220-4553-aa44-4d1f92f549e1 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:03:56.133147 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-71b604d1-8220-4553-aa44-4d1f92f549e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:03:56.148666 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:56.155571 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bda9a5-8c3c-4f57-81a2-1f17cb357f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:56.156605 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bda9a5-8c3c-4f57-81a2-1f17cb357f23 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:56.157445 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bda9a5-8c3c-4f57-81a2-1f17cb357f23 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:56.157811 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bda9a5-8c3c-4f57-81a2-1f17cb357f23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:56.171755 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:56.172937 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bda9a5-8c3c-4f57-81a2-1f17cb357f23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:56.174879 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bda9a5-8c3c-4f57-81a2-1f17cb357f23 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:56.196090 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-c6bda9a5-8c3c-4f57-81a2-1f17cb357f23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:56.196090 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:56.196090 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:56.196090 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:56.196090 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:56.196090 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:56.196090 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:56.196090 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:56.196090 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:56.196090 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:56.196090 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:56.196090 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:56.196090 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:03:56.196090 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:03:56.196090 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:03:56.196090 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:03:56.197165 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:56.197165 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:56.197165 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:56.197165 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:56.197165 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:56.197165 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:56.197165 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:56.197165 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:56.197165 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:56.197165 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:56.197165 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:56.197165 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:56.197165 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:56.197165 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:56.197165 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:56.197165 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:56.197165 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:03:56.198038 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:03:56.198038 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:56.198038 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:56.198038 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 431/881] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:03:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:56.204323 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bda9a5-8c3c-4f57-81a2-1f17cb357f23 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:56.219597 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 451/882] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:03:56 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:03:56.311834 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bda9a5-8c3c-4f57-81a2-1f17cb357f23 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:56.346422 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6016e9-8f5b-4f24-bb98-3bdc95507347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:56.347031 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6016e9-8f5b-4f24-bb98-3bdc95507347 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:56.347351 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6016e9-8f5b-4f24-bb98-3bdc95507347 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:56.347627 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6016e9-8f5b-4f24-bb98-3bdc95507347 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:56.348263 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6016e9-8f5b-4f24-bb98-3bdc95507347 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:56.348554 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6016e9-8f5b-4f24-bb98-3bdc95507347 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:56.349961 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6016e9-8f5b-4f24-bb98-3bdc95507347 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:56.362403 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 432/883] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:03:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:56.368046 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b604d1-8220-4553-aa44-4d1f92f549e1 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:56.376793 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6016e9-8f5b-4f24-bb98-3bdc95507347 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:56.431256 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:56.492591 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b604d1-8220-4553-aa44-4d1f92f549e1 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:56.497395 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6016e9-8f5b-4f24-bb98-3bdc95507347 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:56.529887 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c77cba-8023-487d-b094-3448746a8f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:56.530523 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c77cba-8023-487d-b094-3448746a8f14 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:56.530797 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c77cba-8023-487d-b094-3448746a8f14 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:56.531063 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c77cba-8023-487d-b094-3448746a8f14 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:56.538329 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c77cba-8023-487d-b094-3448746a8f14 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:56.539763 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c77cba-8023-487d-b094-3448746a8f14 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:56.539935 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21929154-e73b-4e30-b344-94bf5e2db6db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:56.540787 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21929154-e73b-4e30-b344-94bf5e2db6db None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:56.541485 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21929154-e73b-4e30-b344-94bf5e2db6db None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:56.541931 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21929154-e73b-4e30-b344-94bf5e2db6db None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:56.543251 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21929154-e73b-4e30-b344-94bf5e2db6db None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:56.544844 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21929154-e73b-4e30-b344-94bf5e2db6db None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:56.551197 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-e6c77cba-8023-487d-b094-3448746a8f14 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 16:03:56.551197 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:56.551197 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:56.551197 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:56.551197 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:56.551197 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:56.551197 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:56.551197 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:56.551197 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:56.551197 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:56.551197 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:56.551197 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:56.551197 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:03:56.551197 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:03:56.551197 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:03:56.551197 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:03:56.552147 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:56.552147 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:56.552147 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:03:56.552147 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:03:56.552147 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:03:56.552147 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:03:56.552147 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:56.552147 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:56.552147 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:56.552147 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:56.552147 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:56.552147 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:56.552147 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:56.552147 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:56.552147 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:56.552147 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:56.552147 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:56.552147 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:56.553605 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:56.553605 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:56.553605 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:03:56.553605 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:03:56.553605 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:03:56.553605 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:03:56.553605 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:03:56.553605 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:03:56.553605 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 16:03:56.553605 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:56.553605 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 452/884] 10.222.0.43 () {56 vars in 1117 bytes} [Tue Sep 6 16:03:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:56.553605 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c77cba-8023-487d-b094-3448746a8f14 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:56.559396 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-21929154-e73b-4e30-b344-94bf5e2db6db None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 16:03:56.559396 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:56.559396 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 16:03:56.559396 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 16:03:56.559396 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 16:03:56.559396 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 16:03:56.559396 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 16:03:56.559396 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:03:56.559396 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 16:03:56.559396 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:56.559396 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 16:03:56.559396 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:56.559396 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:56.559396 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:56.559396 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:56.559396 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:56.559396 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:56.560376 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:56.560376 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:56.560376 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:56.560376 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:56.560376 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:56.560376 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:56.560376 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 16:03:56.560376 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 16:03:56.560376 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 16:03:56.560376 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 16:03:56.560376 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:56.560376 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:56.560376 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:56.560376 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:56.560376 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:56.560376 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:56.561273 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:56.561273 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:56.561273 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:56.561273 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:56.561273 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:56.561273 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:56.561273 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:56.561273 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:56.561273 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:56.561273 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:56.561273 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 16:03:56.561273 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:03:56.561273 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 16:03:56.561273 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:03:56.565362 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 433/885] 10.222.0.43 () {56 vars in 1114 bytes} [Tue Sep 6 16:03:56 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:56.566210 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21929154-e73b-4e30-b344-94bf5e2db6db None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:56.586222 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:56.660897 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c77cba-8023-487d-b094-3448746a8f14 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:56.672275 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21929154-e73b-4e30-b344-94bf5e2db6db None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:56.682080 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:56.696315 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80274a84-77af-4278-8e8d-caf29201fdac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:56.696975 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80274a84-77af-4278-8e8d-caf29201fdac None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:56.697249 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80274a84-77af-4278-8e8d-caf29201fdac None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:56.697523 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80274a84-77af-4278-8e8d-caf29201fdac None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:56.698095 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80274a84-77af-4278-8e8d-caf29201fdac None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:56.699466 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80274a84-77af-4278-8e8d-caf29201fdac None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:56.708533 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-80274a84-77af-4278-8e8d-caf29201fdac None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 16:03:56.708533 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:56.708533 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 16:03:56.708533 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 16:03:56.708533 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 16:03:56.708533 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 16:03:56.708533 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 16:03:56.708533 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:03:56.708533 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 16:03:56.708533 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:56.708533 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 16:03:56.708533 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:56.708533 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:56.708533 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:56.708533 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:56.708533 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:56.708533 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:56.709569 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:56.709569 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:56.709569 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:56.709569 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:56.709569 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:56.709569 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:56.709569 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 16:03:56.709569 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 16:03:56.709569 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 16:03:56.709569 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 16:03:56.709569 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:56.709569 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:56.709569 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:56.709569 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:56.709569 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:56.709569 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:56.710400 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:56.710400 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:56.710400 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:56.710400 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:56.710400 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:56.710400 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:56.710400 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:56.710400 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:56.710400 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:56.710400 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:56.710400 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 16:03:56.710400 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:03:56.710400 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 16:03:56.710400 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:56.710400 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 453/886] 10.222.0.43 () {56 vars in 1126 bytes} [Tue Sep 6 16:03:56 2022] GET /identity/v3/services/object-store => generated 93 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:56.711611 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80274a84-77af-4278-8e8d-caf29201fdac None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:56.714739 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef08868-54ec-4189-b338-3f4fe57b3bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:56.715592 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef08868-54ec-4189-b338-3f4fe57b3bdd None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:56.715872 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef08868-54ec-4189-b338-3f4fe57b3bdd None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:56.716143 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef08868-54ec-4189-b338-3f4fe57b3bdd None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:56.716929 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef08868-54ec-4189-b338-3f4fe57b3bdd None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:56.717233 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef08868-54ec-4189-b338-3f4fe57b3bdd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:56.718666 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef08868-54ec-4189-b338-3f4fe57b3bdd None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:56.737719 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 434/887] 10.222.0.43 () {56 vars in 1149 bytes} [Tue Sep 6 16:03:56 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:56.738541 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef08868-54ec-4189-b338-3f4fe57b3bdd None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:56.811436 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:56.820769 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80274a84-77af-4278-8e8d-caf29201fdac None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:56.838552 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef08868-54ec-4189-b338-3f4fe57b3bdd None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:56.855141 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c595bef6-c882-482a-bd6c-537291385d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:56.855755 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c595bef6-c882-482a-bd6c-537291385d1d None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:56.856031 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c595bef6-c882-482a-bd6c-537291385d1d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:56.856303 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c595bef6-c882-482a-bd6c-537291385d1d None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:56.856960 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c595bef6-c882-482a-bd6c-537291385d1d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:56.857260 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c595bef6-c882-482a-bd6c-537291385d1d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:56.858633 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c595bef6-c882-482a-bd6c-537291385d1d None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:56.863334 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 454/888] 10.222.0.43 () {56 vars in 1114 bytes} [Tue Sep 6 16:03:56 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:56.864109 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c595bef6-c882-482a-bd6c-537291385d1d None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:56.873145 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f9bfa6-95b9-4b00-85fa-2f5dea847297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:56.873758 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f9bfa6-95b9-4b00-85fa-2f5dea847297 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:56.874036 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f9bfa6-95b9-4b00-85fa-2f5dea847297 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:56.874308 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f9bfa6-95b9-4b00-85fa-2f5dea847297 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:56.874937 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f9bfa6-95b9-4b00-85fa-2f5dea847297 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:56.875232 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f9bfa6-95b9-4b00-85fa-2f5dea847297 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:56.876609 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f9bfa6-95b9-4b00-85fa-2f5dea847297 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:56.881151 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 435/889] 10.222.0.43 () {56 vars in 1122 bytes} [Tue Sep 6 16:03:56 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:56.882271 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286fa962-67b1-4a65-8760-7a749f9140e1 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:56.882551 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286fa962-67b1-4a65-8760-7a749f9140e1 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:56.882821 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286fa962-67b1-4a65-8760-7a749f9140e1 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:56.883997 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 436/890] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:03:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:03:56.884684 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286fa962-67b1-4a65-8760-7a749f9140e1 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:56.915459 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:56.965545 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c595bef6-c882-482a-bd6c-537291385d1d None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:56.985352 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286fa962-67b1-4a65-8760-7a749f9140e1 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:56.997346 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64eccaa-6329-44a3-bbb6-52c7f219960c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:56.997950 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64eccaa-6329-44a3-bbb6-52c7f219960c None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:56.998222 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64eccaa-6329-44a3-bbb6-52c7f219960c None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:56.998490 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64eccaa-6329-44a3-bbb6-52c7f219960c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:57.019110 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff09b81-8aca-4a47-a7b5-3bfe490553da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:57.019702 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff09b81-8aca-4a47-a7b5-3bfe490553da None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:57.019986 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff09b81-8aca-4a47-a7b5-3bfe490553da None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:57.020258 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff09b81-8aca-4a47-a7b5-3bfe490553da None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:57.020917 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff09b81-8aca-4a47-a7b5-3bfe490553da None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:57.021218 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff09b81-8aca-4a47-a7b5-3bfe490553da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:57.022094 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:57.022595 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff09b81-8aca-4a47-a7b5-3bfe490553da None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:57.027658 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 437/891] 10.222.0.43 () {56 vars in 1114 bytes} [Tue Sep 6 16:03:56 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:57.028435 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff09b81-8aca-4a47-a7b5-3bfe490553da None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:57.066989 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64eccaa-6329-44a3-bbb6-52c7f219960c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:57.068368 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64eccaa-6329-44a3-bbb6-52c7f219960c None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:57.120623 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:57.127749 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff09b81-8aca-4a47-a7b5-3bfe490553da None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:57.160927 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89322754-2fdd-410c-9053-e68663ef87b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:57.161519 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89322754-2fdd-410c-9053-e68663ef87b7 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:57.161795 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89322754-2fdd-410c-9053-e68663ef87b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:57.162063 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89322754-2fdd-410c-9053-e68663ef87b7 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:57.162684 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89322754-2fdd-410c-9053-e68663ef87b7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:57.162962 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 455/892] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 6 16:03:56 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 16:03:57.163050 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89322754-2fdd-410c-9053-e68663ef87b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:57.164193 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02258132-231a-4404-ba1d-c619b28f45ef None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:57.164414 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89322754-2fdd-410c-9053-e68663ef87b7 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:57.164509 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02258132-231a-4404-ba1d-c619b28f45ef None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:57.164725 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02258132-231a-4404-ba1d-c619b28f45ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:57.180783 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 438/893] 10.222.0.43 () {56 vars in 1122 bytes} [Tue Sep 6 16:03:57 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:57.180783 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89322754-2fdd-410c-9053-e68663ef87b7 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:57.197518 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-02258132-231a-4404-ba1d-c619b28f45ef None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:03:57.198924 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-02258132-231a-4404-ba1d-c619b28f45ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:03:57.259729 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:57.281972 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 456/894] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:03:57 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:03:57.296828 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02258132-231a-4404-ba1d-c619b28f45ef None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:57.297043 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89322754-2fdd-410c-9053-e68663ef87b7 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:57.323287 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525ed25a-473d-443b-b6c9-3a03a70129e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:57.323878 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ed25a-473d-443b-b6c9-3a03a70129e8 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:57.324147 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ed25a-473d-443b-b6c9-3a03a70129e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:57.324424 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ed25a-473d-443b-b6c9-3a03a70129e8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:57.325157 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525ed25a-473d-443b-b6c9-3a03a70129e8 None admin] RBAC: Adding query filter params (interface=public, service_id=00d4087f339b4a50adb1fa52a389eaa2, region_id=RegionOne) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:57.325456 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525ed25a-473d-443b-b6c9-3a03a70129e8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:57.326841 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525ed25a-473d-443b-b6c9-3a03a70129e8 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:57.333758 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 439/895] 10.222.0.43 () {56 vars in 1251 bytes} [Tue Sep 6 16:03:57 2022] GET /identity/v3/endpoints?service_id=00d4087f339b4a50adb1fa52a389eaa2&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:57.341097 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525ed25a-473d-443b-b6c9-3a03a70129e8 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:57.372300 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:57.414576 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02258132-231a-4404-ba1d-c619b28f45ef None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:57.455495 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617a218a-ece8-4b77-a1f2-719ac5712777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:57.456383 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617a218a-ece8-4b77-a1f2-719ac5712777 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:57.456661 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617a218a-ece8-4b77-a1f2-719ac5712777 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:57.457373 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617a218a-ece8-4b77-a1f2-719ac5712777 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:57.459942 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525ed25a-473d-443b-b6c9-3a03a70129e8 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:57.469631 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617a218a-ece8-4b77-a1f2-719ac5712777 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:57.471732 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617a218a-ece8-4b77-a1f2-719ac5712777 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:57.491349 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-617a218a-ece8-4b77-a1f2-719ac5712777 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:57.491349 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:57.491349 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:57.491349 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:57.491349 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:57.491349 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:57.491349 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:57.491349 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:57.491349 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:57.491349 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:57.491349 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:57.491349 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:57.491349 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:03:57.491349 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:03:57.491349 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:03:57.491349 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:03:57.492317 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:57.492317 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:57.492317 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:57.492317 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:57.492317 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:57.492317 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:57.492317 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:57.492317 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:57.492317 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:57.492317 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:57.492317 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:57.492317 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:57.492317 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:57.492317 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:57.492317 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:57.492317 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:57.492317 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:03:57.493260 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:03:57.493260 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:03:57.493260 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:57.493260 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 457/896] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:03:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:57.499045 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617a218a-ece8-4b77-a1f2-719ac5712777 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:57.509626 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad3b9d3-4c6e-46b0-aa08-eaafbcb62903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:57.510750 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad3b9d3-4c6e-46b0-aa08-eaafbcb62903 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:57.511099 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad3b9d3-4c6e-46b0-aa08-eaafbcb62903 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:57.511478 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad3b9d3-4c6e-46b0-aa08-eaafbcb62903 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:57.512491 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad3b9d3-4c6e-46b0-aa08-eaafbcb62903 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:57.514493 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad3b9d3-4c6e-46b0-aa08-eaafbcb62903 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:57.524868 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 440/897] 10.222.0.43 () {56 vars in 1087 bytes} [Tue Sep 6 16:03:57 2022] GET /identity/v3/services => generated 1011 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 16:03:57.550307 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad3b9d3-4c6e-46b0-aa08-eaafbcb62903 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:57.551631 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:57.644328 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617a218a-ece8-4b77-a1f2-719ac5712777 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:57.647216 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:57.665766 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad3b9d3-4c6e-46b0-aa08-eaafbcb62903 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:57.694138 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12604918-4720-4c59-ac8a-d821a7eeb03a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:57.695031 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12604918-4720-4c59-ac8a-d821a7eeb03a None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:57.695322 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12604918-4720-4c59-ac8a-d821a7eeb03a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:57.695595 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12604918-4720-4c59-ac8a-d821a7eeb03a None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:57.696248 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12604918-4720-4c59-ac8a-d821a7eeb03a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:57.696549 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12604918-4720-4c59-ac8a-d821a7eeb03a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:57.698536 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12604918-4720-4c59-ac8a-d821a7eeb03a None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:57.710285 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a835ea-ee9b-4809-b720-e0aaf70c8ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:57.711544 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a835ea-ee9b-4809-b720-e0aaf70c8ce6 None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:57.712032 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a835ea-ee9b-4809-b720-e0aaf70c8ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:57.712535 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a835ea-ee9b-4809-b720-e0aaf70c8ce6 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:57.713486 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a835ea-ee9b-4809-b720-e0aaf70c8ce6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:57.715061 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a835ea-ee9b-4809-b720-e0aaf70c8ce6 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:57.715822 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 458/898] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:03:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:03:57.717093 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12604918-4720-4c59-ac8a-d821a7eeb03a None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:57.744814 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 441/899] 10.222.0.43 () {60 vars in 1141 bytes} [Tue Sep 6 16:03:57 2022] POST /identity/v3/services => generated 269 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:03:57.829518 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:57.834653 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12604918-4720-4c59-ac8a-d821a7eeb03a None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:57.877639 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be7611e-33a1-48f2-a50d-60485baa8104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:57.878572 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be7611e-33a1-48f2-a50d-60485baa8104 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:57.879013 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be7611e-33a1-48f2-a50d-60485baa8104 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:57.879427 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be7611e-33a1-48f2-a50d-60485baa8104 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:57.886832 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be7611e-33a1-48f2-a50d-60485baa8104 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:57.888538 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be7611e-33a1-48f2-a50d-60485baa8104 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:57.900313 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-1be7611e-33a1-48f2-a50d-60485baa8104 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:03:57.900313 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:57.900313 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:57.900313 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:57.900313 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:57.900313 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:57.900313 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:57.900313 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:57.900313 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:57.900313 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:57.900313 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:57.900313 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:57.900313 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:03:57.900313 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:03:57.900313 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:03:57.901381 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:03:57.901381 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:57.901381 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:57.901381 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:57.901381 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:57.901381 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:57.901381 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:57.901381 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:57.901381 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:57.901381 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:57.901381 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:57.901381 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:57.901381 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:57.901381 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:57.901381 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:57.901381 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:57.901381 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:57.902465 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:03:57.902465 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:03:57.902465 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:03:57.902465 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:03:57.902465 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:03:57.902465 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:03:57.902465 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:03:57.902465 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:03:57.907527 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 459/900] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 6 16:03:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:03:57.910742 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a835ea-ee9b-4809-b720-e0aaf70c8ce6 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:58.014698 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a835ea-ee9b-4809-b720-e0aaf70c8ce6 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:58.051010 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d139c57c-0fd7-4d8a-81a3-902930d8dfe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:58.051902 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d139c57c-0fd7-4d8a-81a3-902930d8dfe4 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:58.052339 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d139c57c-0fd7-4d8a-81a3-902930d8dfe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:58.052772 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d139c57c-0fd7-4d8a-81a3-902930d8dfe4 None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:58.053677 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d139c57c-0fd7-4d8a-81a3-902930d8dfe4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:58.054129 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d139c57c-0fd7-4d8a-81a3-902930d8dfe4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:58.055715 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d139c57c-0fd7-4d8a-81a3-902930d8dfe4 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:58.072867 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 442/901] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 6 16:03:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:58.075723 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be7611e-33a1-48f2-a50d-60485baa8104 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:58.182508 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be7611e-33a1-48f2-a50d-60485baa8104 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:58.220241 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbef8421-a952-49a6-8978-824bd4221c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:58.221998 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbef8421-a952-49a6-8978-824bd4221c53 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:58.221998 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbef8421-a952-49a6-8978-824bd4221c53 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:58.221998 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbef8421-a952-49a6-8978-824bd4221c53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:58.222253 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbef8421-a952-49a6-8978-824bd4221c53 None admin] RBAC: Adding query filter params (role.id=012f11d05b68448696f577c7ab2161a8, scope.project.id=128d210e8dad48ac921ed4cb1d444ceb, user.id=d865006413b24ec3922619311f81f353) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:58.222524 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbef8421-a952-49a6-8978-824bd4221c53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:58.223934 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbef8421-a952-49a6-8978-824bd4221c53 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:58.229854 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 460/902] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 6 16:03:58 2022] GET /identity/v3/role_assignments?user.id=d865006413b24ec3922619311f81f353&scope.project.id=128d210e8dad48ac921ed4cb1d444ceb&role.id=012f11d05b68448696f577c7ab2161a8 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:59.143112 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e1ae40-6870-41c5-9938-74631e9a7094 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:59.143650 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e1ae40-6870-41c5-9938-74631e9a7094 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:59.143726 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e1ae40-6870-41c5-9938-74631e9a7094 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:59.145345 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 443/903] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:03:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:03:59.150503 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a33632d-e4b3-4dc9-9357-c6d8199866d3 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:59.150938 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a33632d-e4b3-4dc9-9357-c6d8199866d3 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:59.151084 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a33632d-e4b3-4dc9-9357-c6d8199866d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:59.169700 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-2a33632d-e4b3-4dc9-9357-c6d8199866d3 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:03:59.170881 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-2a33632d-e4b3-4dc9-9357-c6d8199866d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:03:59.185135 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:59.251088 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 461/904] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:03:59 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:03:59.517011 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e1ae40-6870-41c5-9938-74631e9a7094 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:59.527101 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:59.545802 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb86bacb-86be-4350-bc41-6692f065995d None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:59.546136 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb86bacb-86be-4350-bc41-6692f065995d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:59.546330 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb86bacb-86be-4350-bc41-6692f065995d None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:59.547927 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 462/905] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:03:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 16:03:59.552908 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285a3195-1476-4b77-810a-4a2f2706fc42 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:59.553315 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285a3195-1476-4b77-810a-4a2f2706fc42 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:59.553588 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285a3195-1476-4b77-810a-4a2f2706fc42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:59.572289 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-285a3195-1476-4b77-810a-4a2f2706fc42 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:03:59.573494 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-285a3195-1476-4b77-810a-4a2f2706fc42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:03:59.634615 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e1ae40-6870-41c5-9938-74631e9a7094 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:59.649104 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 463/906] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:03:59 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 16:03:59.670214 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53d0efb-4e06-4abc-8ef7-8ef270f9c073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:59.670823 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53d0efb-4e06-4abc-8ef7-8ef270f9c073 None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:59.671096 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53d0efb-4e06-4abc-8ef7-8ef270f9c073 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:59.671360 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53d0efb-4e06-4abc-8ef7-8ef270f9c073 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:59.672020 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53d0efb-4e06-4abc-8ef7-8ef270f9c073 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:59.673472 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53d0efb-4e06-4abc-8ef7-8ef270f9c073 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:59.693106 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 444/907] 10.222.0.43 () {60 vars in 1141 bytes} [Tue Sep 6 16:03:59 2022] POST /identity/v3/services => generated 241 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:03:59.699731 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a9b3fe-74ef-40b5-a91a-05610ddd7706 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:59.700018 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a9b3fe-74ef-40b5-a91a-05610ddd7706 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:59.700279 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a9b3fe-74ef-40b5-a91a-05610ddd7706 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:59.701908 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 464/908] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:03:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:03:59.710910 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7861e521-f0cf-4179-a8a8-68fa0b833671 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:59.711430 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7861e521-f0cf-4179-a8a8-68fa0b833671 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:59.711850 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7861e521-f0cf-4179-a8a8-68fa0b833671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:59.731998 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-7861e521-f0cf-4179-a8a8-68fa0b833671 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:03:59.733713 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-7861e521-f0cf-4179-a8a8-68fa0b833671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:03:59.801837 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:59.817326 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 445/909] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:03:59 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:03:59.933311 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a9b3fe-74ef-40b5-a91a-05610ddd7706 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:00.013976 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:00.036853 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a9b3fe-74ef-40b5-a91a-05610ddd7706 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:00.072858 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0622ce-317d-44cf-bcf3-a13a5a5795ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:00.073425 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0622ce-317d-44cf-bcf3-a13a5a5795ad None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:00.073698 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0622ce-317d-44cf-bcf3-a13a5a5795ad None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:00.073964 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0622ce-317d-44cf-bcf3-a13a5a5795ad None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:00.074597 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0622ce-317d-44cf-bcf3-a13a5a5795ad None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:00.076023 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0622ce-317d-44cf-bcf3-a13a5a5795ad None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:00.086865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-8b0622ce-317d-44cf-bcf3-a13a5a5795ad None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 16:04:00.086865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:00.086865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 16:04:00.086865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 16:04:00.086865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 16:04:00.086865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 16:04:00.086865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 16:04:00.086865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:04:00.086865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 16:04:00.086865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:04:00.086865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 16:04:00.086865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:04:00.086865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:00.086865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:00.086865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:00.086865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:00.086865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:00.087953 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:00.087953 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:00.087953 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:00.087953 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:00.087953 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:00.087953 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:00.087953 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 16:04:00.087953 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 16:04:00.087953 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 16:04:00.087953 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 16:04:00.087953 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:00.087953 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:00.087953 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:00.087953 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:00.087953 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:00.087953 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:00.088923 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:00.088923 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:00.088923 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:00.088923 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:00.088923 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:00.088923 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:00.088923 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:00.088923 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:00.088923 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:00.088923 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:00.088923 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 16:04:00.088923 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:04:00.088923 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 16:04:00.088923 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:04:00.088923 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 465/910] 10.222.0.43 () {56 vars in 1114 bytes} [Tue Sep 6 16:03:59 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:04:00.091221 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7861e521-f0cf-4179-a8a8-68fa0b833671 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:00.093900 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0622ce-317d-44cf-bcf3-a13a5a5795ad None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:00.197383 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0622ce-317d-44cf-bcf3-a13a5a5795ad None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:00.216801 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7861e521-f0cf-4179-a8a8-68fa0b833671 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:00.233229 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f433d5-e880-4da8-8e3c-2a3dea464bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:00.233935 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f433d5-e880-4da8-8e3c-2a3dea464bd2 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:00.234319 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f433d5-e880-4da8-8e3c-2a3dea464bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:00.234603 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f433d5-e880-4da8-8e3c-2a3dea464bd2 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:00.235239 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f433d5-e880-4da8-8e3c-2a3dea464bd2 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:00.235540 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f433d5-e880-4da8-8e3c-2a3dea464bd2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:00.237017 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f433d5-e880-4da8-8e3c-2a3dea464bd2 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:00.246281 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 466/911] 10.222.0.43 () {56 vars in 1114 bytes} [Tue Sep 6 16:04:00 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:04:00.246281 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe84782-bfe7-43da-93e1-549d65e2001a None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:00.246281 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe84782-bfe7-43da-93e1-549d65e2001a None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:00.246281 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe84782-bfe7-43da-93e1-549d65e2001a None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:00.246281 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 467/912] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:04:00 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:04:00.247767 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe84782-bfe7-43da-93e1-549d65e2001a None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:00.258265 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:00.260491 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1132c5d2-7c33-4022-8fba-60df2fdbbbad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:00.261551 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1132c5d2-7c33-4022-8fba-60df2fdbbbad None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:00.261824 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1132c5d2-7c33-4022-8fba-60df2fdbbbad None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:00.262094 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1132c5d2-7c33-4022-8fba-60df2fdbbbad None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:00.263000 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1132c5d2-7c33-4022-8fba-60df2fdbbbad None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:00.264674 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1132c5d2-7c33-4022-8fba-60df2fdbbbad None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:00.276166 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-1132c5d2-7c33-4022-8fba-60df2fdbbbad None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 16:04:00.276166 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:00.276166 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 16:04:00.276166 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 16:04:00.276166 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 16:04:00.276166 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 16:04:00.276166 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 16:04:00.276166 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:04:00.276166 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 16:04:00.276166 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:04:00.276166 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 16:04:00.276166 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:04:00.276166 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:00.276166 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:00.276166 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:00.276166 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:00.277153 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:00.277153 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:00.277153 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:00.277153 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:00.277153 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:00.277153 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:00.277153 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:00.277153 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 16:04:00.277153 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 16:04:00.277153 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 16:04:00.277153 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 16:04:00.277153 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:00.277153 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:00.277153 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:00.277153 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:00.277153 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:00.277153 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:00.284011 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:00.284011 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:00.284011 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:00.284011 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:00.284011 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:00.284011 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:00.284011 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:00.284011 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:00.284011 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:00.284011 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:00.284011 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 16:04:00.284011 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:04:00.284011 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 16:04:00.284011 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:04:00.284011 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 446/913] 10.222.0.43 () {56 vars in 1132 bytes} [Tue Sep 6 16:04:00 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:04:00.284848 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f752639-4cce-4eca-a150-1284998dd7f5 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:00.284848 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f752639-4cce-4eca-a150-1284998dd7f5 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:00.284848 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f752639-4cce-4eca-a150-1284998dd7f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:00.303828 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-1f752639-4cce-4eca-a150-1284998dd7f5 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:04:00.305039 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1f752639-4cce-4eca-a150-1284998dd7f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:04:00.360863 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:00.361720 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe84782-bfe7-43da-93e1-549d65e2001a None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:00.379240 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 447/914] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:04:00 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:04:00.381627 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f752639-4cce-4eca-a150-1284998dd7f5 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:00.397861 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae927a08-e153-44f7-ad3f-05562102e18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:00.398400 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae927a08-e153-44f7-ad3f-05562102e18e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:00.398655 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae927a08-e153-44f7-ad3f-05562102e18e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:00.398925 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae927a08-e153-44f7-ad3f-05562102e18e None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:00.399599 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae927a08-e153-44f7-ad3f-05562102e18e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:00.399904 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae927a08-e153-44f7-ad3f-05562102e18e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:00.401337 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae927a08-e153-44f7-ad3f-05562102e18e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:00.411120 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 468/915] 10.222.0.43 () {56 vars in 1114 bytes} [Tue Sep 6 16:04:00 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:04:00.411947 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae927a08-e153-44f7-ad3f-05562102e18e None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:00.496457 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f752639-4cce-4eca-a150-1284998dd7f5 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:00.497119 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:00.533558 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55c7003-1225-46ac-82eb-73131a987959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:00.534091 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55c7003-1225-46ac-82eb-73131a987959 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:00.534408 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55c7003-1225-46ac-82eb-73131a987959 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:00.534709 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55c7003-1225-46ac-82eb-73131a987959 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:00.535388 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55c7003-1225-46ac-82eb-73131a987959 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:00.535644 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55c7003-1225-46ac-82eb-73131a987959 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:00.537052 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55c7003-1225-46ac-82eb-73131a987959 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:00.537841 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae927a08-e153-44f7-ad3f-05562102e18e None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:00.546674 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 448/916] 10.222.0.43 () {56 vars in 1126 bytes} [Tue Sep 6 16:04:00 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:00.549533 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55c7003-1225-46ac-82eb-73131a987959 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:00.585947 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b091a60d-6f2f-47fb-9cc2-48273745a748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:00.587725 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b091a60d-6f2f-47fb-9cc2-48273745a748 None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:00.588226 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b091a60d-6f2f-47fb-9cc2-48273745a748 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:00.588689 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b091a60d-6f2f-47fb-9cc2-48273745a748 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:00.589587 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b091a60d-6f2f-47fb-9cc2-48273745a748 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:00.591164 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b091a60d-6f2f-47fb-9cc2-48273745a748 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:00.602345 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:00.622637 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 469/917] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:04:00 2022] POST /identity/v3/endpoints => generated 346 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:04:00.646245 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b091a60d-6f2f-47fb-9cc2-48273745a748 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:00.666447 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55c7003-1225-46ac-82eb-73131a987959 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:00.726953 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cc5edb-fc78-4acb-a8a9-b12dc667299b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:00.727738 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cc5edb-fc78-4acb-a8a9-b12dc667299b None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:00.728123 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cc5edb-fc78-4acb-a8a9-b12dc667299b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:00.728554 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cc5edb-fc78-4acb-a8a9-b12dc667299b None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:00.729653 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cc5edb-fc78-4acb-a8a9-b12dc667299b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:00.729964 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cc5edb-fc78-4acb-a8a9-b12dc667299b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:00.731703 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cc5edb-fc78-4acb-a8a9-b12dc667299b None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:00.739327 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 449/918] 10.222.0.43 () {56 vars in 1126 bytes} [Tue Sep 6 16:04:00 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:00.745896 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cc5edb-fc78-4acb-a8a9-b12dc667299b None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:00.790735 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b091a60d-6f2f-47fb-9cc2-48273745a748 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:00.822436 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:00.836612 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcfa8e9-5f75-4892-b8c9-5dbb661ef5ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:00.838311 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcfa8e9-5f75-4892-b8c9-5dbb661ef5ca None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:00.838583 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcfa8e9-5f75-4892-b8c9-5dbb661ef5ca None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:00.838848 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcfa8e9-5f75-4892-b8c9-5dbb661ef5ca None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:00.856093 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcfa8e9-5f75-4892-b8c9-5dbb661ef5ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:04:00.862841 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-2dcfa8e9-5f75-4892-b8c9-5dbb661ef5ca None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 16:04:00.862841 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:00.862841 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:00.862841 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:00.862841 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:00.862841 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:00.862841 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:00.862841 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:00.862841 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:00.862841 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:00.862841 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:00.862841 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:00.862841 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:04:00.862841 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:04:00.862841 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:04:00.862841 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:04:00.863780 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:04:00.863780 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:04:00.863780 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:04:00.863780 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:04:00.863780 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:00.863780 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:00.863780 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:00.863780 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:00.863780 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:00.863780 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:00.863780 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:00.863780 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:00.863780 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:00.863780 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:00.863780 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:00.863780 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:00.864626 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:00.864626 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:00.864626 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:00.864626 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:00.864626 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:04:00.864626 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:04:00.864626 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:04:00.864626 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:04:00.864626 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:04:00.864626 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:04:00.864626 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 16:04:00.864626 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:04:00.864626 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 470/919] 10.222.0.43 () {56 vars in 1120 bytes} [Tue Sep 6 16:04:00 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:04:00.870353 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcfa8e9-5f75-4892-b8c9-5dbb661ef5ca None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:00.883242 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cc5edb-fc78-4acb-a8a9-b12dc667299b None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:00.897669 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:00.929535 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c3b149-7b07-4694-9af7-5b10756174f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:00.931146 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c3b149-7b07-4694-9af7-5b10756174f7 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:00.932031 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c3b149-7b07-4694-9af7-5b10756174f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:00.932964 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c3b149-7b07-4694-9af7-5b10756174f7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:00.934113 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c3b149-7b07-4694-9af7-5b10756174f7 None admin] RBAC: Adding query filter params (interface=public, service_id=73e14c2464bf4c03826654066d719dbd, region_id=RegionOne) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:00.934605 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c3b149-7b07-4694-9af7-5b10756174f7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:00.937087 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c3b149-7b07-4694-9af7-5b10756174f7 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:00.944435 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 450/920] 10.222.0.43 () {56 vars in 1251 bytes} [Tue Sep 6 16:04:00 2022] GET /identity/v3/endpoints?service_id=73e14c2464bf4c03826654066d719dbd&interface=public®ion_id=RegionOne => generated 194 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:00.949534 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c3b149-7b07-4694-9af7-5b10756174f7 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:00.985149 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcfa8e9-5f75-4892-b8c9-5dbb661ef5ca None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:01.021272 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f33a40-5ddc-4ef5-829f-7284fc0e56f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:01.021883 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f33a40-5ddc-4ef5-829f-7284fc0e56f4 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:01.022164 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f33a40-5ddc-4ef5-829f-7284fc0e56f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:01.022438 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f33a40-5ddc-4ef5-829f-7284fc0e56f4 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:01.023180 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f33a40-5ddc-4ef5-829f-7284fc0e56f4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:01.023399 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f33a40-5ddc-4ef5-829f-7284fc0e56f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:01.024794 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f33a40-5ddc-4ef5-829f-7284fc0e56f4 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:01.038582 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 471/921] 10.222.0.43 () {56 vars in 1115 bytes} [Tue Sep 6 16:04:00 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:04:01.039695 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f33a40-5ddc-4ef5-829f-7284fc0e56f4 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:01.058014 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c3b149-7b07-4694-9af7-5b10756174f7 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:01.092420 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:01.096446 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d041a87-6805-4c47-934b-a7ac9702d5ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:01.097342 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d041a87-6805-4c47-934b-a7ac9702d5ec None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:01.097812 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d041a87-6805-4c47-934b-a7ac9702d5ec None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:01.098266 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d041a87-6805-4c47-934b-a7ac9702d5ec None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:01.099453 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d041a87-6805-4c47-934b-a7ac9702d5ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:01.101597 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d041a87-6805-4c47-934b-a7ac9702d5ec None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:01.113799 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 451/922] 10.222.0.43 () {56 vars in 1087 bytes} [Tue Sep 6 16:04:00 2022] GET /identity/v3/services => generated 1497 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 16:04:01.150421 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f33a40-5ddc-4ef5-829f-7284fc0e56f4 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:01.188263 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b64e334-7da0-49ee-8776-406b1d56417a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:01.189149 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b64e334-7da0-49ee-8776-406b1d56417a None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:01.189473 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b64e334-7da0-49ee-8776-406b1d56417a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:01.189777 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b64e334-7da0-49ee-8776-406b1d56417a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:01.202216 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b64e334-7da0-49ee-8776-406b1d56417a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:01.204232 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b64e334-7da0-49ee-8776-406b1d56417a None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:01.220148 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-4b64e334-7da0-49ee-8776-406b1d56417a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:04:01.220148 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:01.220148 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:01.220148 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:01.220148 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:01.220148 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:01.220148 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:01.220148 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:01.220148 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:01.220148 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:01.220148 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:01.220148 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:01.220148 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:04:01.220148 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:04:01.220148 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:04:01.220148 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:04:01.221346 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:01.221346 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:01.221346 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:01.221346 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:01.221346 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:01.221346 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:01.221346 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:01.221346 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:01.221346 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:01.221346 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:01.221346 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:01.221346 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:01.221346 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:01.221346 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:01.221346 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:01.221346 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:01.221346 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:04:01.222264 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:04:01.222264 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:04:01.222264 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:04:01.222264 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 472/923] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:04:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:04:01.227693 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d041a87-6805-4c47-934b-a7ac9702d5ec None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:01.277660 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:01.339060 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d041a87-6805-4c47-934b-a7ac9702d5ec None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:01.377286 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a77ea4-c7dc-4bf5-9f97-c278d55a3f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:01.377901 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a77ea4-c7dc-4bf5-9f97-c278d55a3f19 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:01.378171 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a77ea4-c7dc-4bf5-9f97-c278d55a3f19 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:01.378436 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a77ea4-c7dc-4bf5-9f97-c278d55a3f19 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:01.379081 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a77ea4-c7dc-4bf5-9f97-c278d55a3f19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:01.379375 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a77ea4-c7dc-4bf5-9f97-c278d55a3f19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:01.380764 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a77ea4-c7dc-4bf5-9f97-c278d55a3f19 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:01.392379 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 452/924] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:04:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:01.399413 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b64e334-7da0-49ee-8776-406b1d56417a None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:01.506189 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:01.509125 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b64e334-7da0-49ee-8776-406b1d56417a None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:01.544535 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c12bc8d-03c9-4f89-aed0-f8865d8aa149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:01.545203 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c12bc8d-03c9-4f89-aed0-f8865d8aa149 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:01.545483 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c12bc8d-03c9-4f89-aed0-f8865d8aa149 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:01.545755 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c12bc8d-03c9-4f89-aed0-f8865d8aa149 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:01.553132 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c12bc8d-03c9-4f89-aed0-f8865d8aa149 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:01.554580 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c12bc8d-03c9-4f89-aed0-f8865d8aa149 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:01.566544 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-4c12bc8d-03c9-4f89-aed0-f8865d8aa149 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 16:04:01.566544 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:01.566544 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:01.566544 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:01.566544 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:01.566544 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:01.566544 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:01.566544 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:01.566544 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:01.566544 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:01.566544 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:01.566544 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:01.566544 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:04:01.566544 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:04:01.566544 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:04:01.566544 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:04:01.567524 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:01.567524 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:01.567524 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:04:01.567524 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:04:01.567524 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:04:01.567524 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:04:01.567524 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:01.567524 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:01.567524 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:01.567524 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:01.567524 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:01.567524 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:01.567524 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:01.567524 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:01.567524 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:01.567524 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:01.567524 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:01.567524 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:01.568446 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:01.568446 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:01.568446 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:04:01.568446 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:04:01.568446 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:04:01.568446 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:04:01.568446 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:04:01.568446 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:04:01.568446 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 16:04:01.568446 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:04:01.568446 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 473/925] 10.222.0.43 () {56 vars in 1117 bytes} [Tue Sep 6 16:04:01 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:04:01.573844 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a77ea4-c7dc-4bf5-9f97-c278d55a3f19 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:01.657360 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363aa345-f438-45c6-8fb6-dfd98de37a59 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:01.657656 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363aa345-f438-45c6-8fb6-dfd98de37a59 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:01.657916 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363aa345-f438-45c6-8fb6-dfd98de37a59 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:01.660515 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 474/926] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:04:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:04:01.664567 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96e408e-6e7e-4351-b7ee-5ff77de58edd None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:01.664890 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96e408e-6e7e-4351-b7ee-5ff77de58edd None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:01.665163 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96e408e-6e7e-4351-b7ee-5ff77de58edd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:01.673284 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:01.676104 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a77ea4-c7dc-4bf5-9f97-c278d55a3f19 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:01.686849 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-b96e408e-6e7e-4351-b7ee-5ff77de58edd None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:04:01.688529 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b96e408e-6e7e-4351-b7ee-5ff77de58edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:04:01.713392 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083adef3-ed54-48f0-87d3-949e0cacf971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:01.714235 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083adef3-ed54-48f0-87d3-949e0cacf971 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:01.714675 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083adef3-ed54-48f0-87d3-949e0cacf971 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:01.715017 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083adef3-ed54-48f0-87d3-949e0cacf971 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:01.715878 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083adef3-ed54-48f0-87d3-949e0cacf971 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:01.716250 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083adef3-ed54-48f0-87d3-949e0cacf971 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:01.718078 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083adef3-ed54-48f0-87d3-949e0cacf971 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:01.737690 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 453/927] 10.222.0.43 () {56 vars in 1149 bytes} [Tue Sep 6 16:04:01 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:01.743740 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083adef3-ed54-48f0-87d3-949e0cacf971 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:01.764220 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 475/928] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:04:01 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:04:01.848312 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083adef3-ed54-48f0-87d3-949e0cacf971 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:01.881471 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28e0272-1cd6-4dc0-8deb-e634c750c829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:01.882096 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28e0272-1cd6-4dc0-8deb-e634c750c829 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:01.882367 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28e0272-1cd6-4dc0-8deb-e634c750c829 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:01.882633 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28e0272-1cd6-4dc0-8deb-e634c750c829 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:01.892716 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28e0272-1cd6-4dc0-8deb-e634c750c829 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:01.894158 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28e0272-1cd6-4dc0-8deb-e634c750c829 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:01.908488 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-d28e0272-1cd6-4dc0-8deb-e634c750c829 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:04:01.908488 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:01.908488 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:01.908488 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:01.908488 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:01.908488 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:01.908488 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:01.908488 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:01.908488 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:01.908488 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:01.908488 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:01.908488 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:01.908488 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:04:01.908488 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:04:01.908488 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:04:01.908488 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:04:01.909613 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:01.909613 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:01.909613 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:01.909613 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:01.909613 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:01.909613 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:01.909613 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:01.909613 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:01.909613 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:01.909613 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:01.909613 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:01.909613 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:01.909613 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:01.909613 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:01.909613 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:01.909613 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:01.909613 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:04:01.910705 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:04:01.910705 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:04:01.910705 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:04:01.910705 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 454/929] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:04:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:04:01.916083 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96e408e-6e7e-4351-b7ee-5ff77de58edd None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:02.008667 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:02.023507 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96e408e-6e7e-4351-b7ee-5ff77de58edd None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:02.031069 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28e0272-1cd6-4dc0-8deb-e634c750c829 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:02.043281 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:02.065939 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1693ddc9-62ca-44b2-8439-bf02006eab38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:02.066963 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1693ddc9-62ca-44b2-8439-bf02006eab38 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:02.067344 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1693ddc9-62ca-44b2-8439-bf02006eab38 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:02.067613 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1693ddc9-62ca-44b2-8439-bf02006eab38 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:02.068253 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1693ddc9-62ca-44b2-8439-bf02006eab38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:02.068548 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1693ddc9-62ca-44b2-8439-bf02006eab38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:02.070119 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1693ddc9-62ca-44b2-8439-bf02006eab38 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:02.086536 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 476/930] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:04:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:02.096472 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1693ddc9-62ca-44b2-8439-bf02006eab38 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:02.150470 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28e0272-1cd6-4dc0-8deb-e634c750c829 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:02.189946 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6735498c-f561-42cb-8e30-b0cec35f58fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:02.190587 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6735498c-f561-42cb-8e30-b0cec35f58fb None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:02.190857 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6735498c-f561-42cb-8e30-b0cec35f58fb None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:02.191120 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6735498c-f561-42cb-8e30-b0cec35f58fb None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:02.191705 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6735498c-f561-42cb-8e30-b0cec35f58fb None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:02.193149 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6735498c-f561-42cb-8e30-b0cec35f58fb None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:02.196425 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1693ddc9-62ca-44b2-8439-bf02006eab38 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:02.202417 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-6735498c-f561-42cb-8e30-b0cec35f58fb None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 16:04:02.202417 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:02.202417 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 16:04:02.202417 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 16:04:02.202417 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 16:04:02.202417 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 16:04:02.202417 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 16:04:02.202417 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:04:02.202417 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 16:04:02.202417 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:04:02.202417 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 16:04:02.202417 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:04:02.202417 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:02.202417 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:02.202417 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:02.202417 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:02.202417 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:02.203969 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:02.203969 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:02.203969 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:02.203969 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:02.203969 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:02.203969 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:02.203969 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 16:04:02.203969 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 16:04:02.203969 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 16:04:02.203969 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 16:04:02.203969 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:02.203969 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:02.203969 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:02.203969 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:02.203969 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:02.203969 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:02.205248 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:02.205248 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:02.205248 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:02.205248 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:02.205248 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:02.205248 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:02.205248 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:02.205248 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:02.205248 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:02.205248 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:02.205248 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 16:04:02.205248 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:04:02.205248 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 16:04:02.205248 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:04:02.205248 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 455/931] 10.222.0.43 () {56 vars in 1126 bytes} [Tue Sep 6 16:04:02 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:04:02.212856 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6735498c-f561-42cb-8e30-b0cec35f58fb None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:02.242123 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4964bae-7679-42eb-bacf-ea8b4c4d47d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:02.242751 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4964bae-7679-42eb-bacf-ea8b4c4d47d1 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:02.243180 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4964bae-7679-42eb-bacf-ea8b4c4d47d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:02.243517 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4964bae-7679-42eb-bacf-ea8b4c4d47d1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:02.254413 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4964bae-7679-42eb-bacf-ea8b4c4d47d1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:02.256008 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4964bae-7679-42eb-bacf-ea8b4c4d47d1 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:02.268726 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-c4964bae-7679-42eb-bacf-ea8b4c4d47d1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:04:02.268726 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:02.268726 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:02.268726 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:02.268726 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:02.268726 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:02.268726 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:02.268726 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:02.268726 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:02.268726 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:02.268726 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:02.268726 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:02.268726 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:04:02.268726 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:04:02.268726 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:04:02.269660 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:04:02.269660 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:02.269660 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:02.269660 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:02.269660 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:02.269660 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:02.269660 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:02.269660 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:02.269660 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:02.269660 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:02.269660 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:02.269660 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:02.269660 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:02.269660 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:02.269660 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:02.269660 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:02.269660 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:02.270860 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:04:02.270860 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:04:02.270860 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:04:02.270860 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:04:02.270860 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:04:02.270860 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:04:02.270860 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:04:02.270860 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:04:02.281180 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 477/932] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 6 16:04:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:04:02.281901 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4964bae-7679-42eb-bacf-ea8b4c4d47d1 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:02.296248 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:02.312794 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:02.358761 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6735498c-f561-42cb-8e30-b0cec35f58fb None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:02.394615 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4964bae-7679-42eb-bacf-ea8b4c4d47d1 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:02.403277 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a48f524-db92-4cfd-808d-a7fdbc1b8709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:02.404554 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a48f524-db92-4cfd-808d-a7fdbc1b8709 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:02.405002 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a48f524-db92-4cfd-808d-a7fdbc1b8709 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:02.405444 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a48f524-db92-4cfd-808d-a7fdbc1b8709 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:02.406658 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a48f524-db92-4cfd-808d-a7fdbc1b8709 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:02.407106 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a48f524-db92-4cfd-808d-a7fdbc1b8709 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:02.409409 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a48f524-db92-4cfd-808d-a7fdbc1b8709 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:02.421221 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 456/933] 10.222.0.43 () {56 vars in 1122 bytes} [Tue Sep 6 16:04:02 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:04:02.423159 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a48f524-db92-4cfd-808d-a7fdbc1b8709 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:02.436227 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fbd299-d398-47fd-9a46-e47ca342d189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:02.437150 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fbd299-d398-47fd-9a46-e47ca342d189 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:02.437263 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fbd299-d398-47fd-9a46-e47ca342d189 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:02.437634 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fbd299-d398-47fd-9a46-e47ca342d189 None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:02.438441 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fbd299-d398-47fd-9a46-e47ca342d189 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:02.438771 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fbd299-d398-47fd-9a46-e47ca342d189 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:02.440336 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fbd299-d398-47fd-9a46-e47ca342d189 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:02.457129 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 478/934] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 6 16:04:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:02.459404 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fbd299-d398-47fd-9a46-e47ca342d189 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:02.530101 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:02.534866 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a48f524-db92-4cfd-808d-a7fdbc1b8709 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:02.563469 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fbd299-d398-47fd-9a46-e47ca342d189 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:02.577520 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8582245-189d-4832-aab9-aa545fecddc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:02.578504 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8582245-189d-4832-aab9-aa545fecddc4 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:02.579364 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8582245-189d-4832-aab9-aa545fecddc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:02.579825 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8582245-189d-4832-aab9-aa545fecddc4 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:02.580839 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8582245-189d-4832-aab9-aa545fecddc4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:02.581431 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8582245-189d-4832-aab9-aa545fecddc4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:02.583005 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8582245-189d-4832-aab9-aa545fecddc4 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:02.585452 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:02.594544 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 457/935] 10.222.0.43 () {56 vars in 1122 bytes} [Tue Sep 6 16:04:02 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:02.597312 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8582245-189d-4832-aab9-aa545fecddc4 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:02.599603 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0d383b-2d19-451f-9b5e-ceacb653f36a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:02.600376 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0d383b-2d19-451f-9b5e-ceacb653f36a None admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:02.600666 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0d383b-2d19-451f-9b5e-ceacb653f36a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:02.600970 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0d383b-2d19-451f-9b5e-ceacb653f36a None admin] PATH_INFO: `/v3/projects/128d210e8dad48ac921ed4cb1d444ceb/users/d865006413b24ec3922619311f81f353/roles/012f11d05b68448696f577c7ab2161a8` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:02.603927 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0d383b-2d19-451f-9b5e-ceacb653f36a None admin] RBAC: Authorizing `identity:create_grant(project_id=128d210e8dad48ac921ed4cb1d444ceb, user_id=d865006413b24ec3922619311f81f353, role_id=012f11d05b68448696f577c7ab2161a8)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:02.605529 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0d383b-2d19-451f-9b5e-ceacb653f36a None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:02.624833 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 479/936] 10.222.0.43 () {58 vars in 1439 bytes} [Tue Sep 6 16:04:02 2022] PUT /identity/v3/projects/128d210e8dad48ac921ed4cb1d444ceb/users/d865006413b24ec3922619311f81f353/roles/012f11d05b68448696f577c7ab2161a8 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:04:02.638431 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb64ee-d99c-4256-84c8-3789927462e1 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:02.639102 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb64ee-d99c-4256-84c8-3789927462e1 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:02.639575 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb64ee-d99c-4256-84c8-3789927462e1 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:02.641533 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 480/937] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:04:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 16:04:02.648552 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b3d78b-2822-409e-889c-bee5af60571d None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:02.648892 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b3d78b-2822-409e-889c-bee5af60571d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:02.649128 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b3d78b-2822-409e-889c-bee5af60571d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:02.668091 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-04b3d78b-2822-409e-889c-bee5af60571d None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:04:02.669325 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-04b3d78b-2822-409e-889c-bee5af60571d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:04:02.732915 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8582245-189d-4832-aab9-aa545fecddc4 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:02.760909 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 481/938] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:04:02 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 16:04:02.781636 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31631859-1674-4a05-9d92-a5063b9fa490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:02.782670 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31631859-1674-4a05-9d92-a5063b9fa490 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:02.783392 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31631859-1674-4a05-9d92-a5063b9fa490 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:02.783918 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31631859-1674-4a05-9d92-a5063b9fa490 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:02.784945 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31631859-1674-4a05-9d92-a5063b9fa490 None admin] RBAC: Adding query filter params (interface=public, service_id=70d749483b0d4dbcbcd119fd4379ae25, region_id=RegionOne) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:02.785522 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31631859-1674-4a05-9d92-a5063b9fa490 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:02.787142 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31631859-1674-4a05-9d92-a5063b9fa490 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:02.797894 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 458/939] 10.222.0.43 () {56 vars in 1251 bytes} [Tue Sep 6 16:04:02 2022] GET /identity/v3/endpoints?service_id=70d749483b0d4dbcbcd119fd4379ae25&interface=public®ion_id=RegionOne => generated 194 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:04:02.800467 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b3d78b-2822-409e-889c-bee5af60571d None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:02.865576 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:02.909960 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b3d78b-2822-409e-889c-bee5af60571d None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:02.944574 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a93cdf-ed25-4663-bd29-1c56c02e599f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:02.945568 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a93cdf-ed25-4663-bd29-1c56c02e599f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:02.946043 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a93cdf-ed25-4663-bd29-1c56c02e599f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:02.946538 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a93cdf-ed25-4663-bd29-1c56c02e599f None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:02.947351 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a93cdf-ed25-4663-bd29-1c56c02e599f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:02.948946 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a93cdf-ed25-4663-bd29-1c56c02e599f None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:02.960885 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 482/940] 10.222.0.43 () {56 vars in 1087 bytes} [Tue Sep 6 16:04:02 2022] GET /identity/v3/services => generated 1497 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:04:03.034100 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31631859-1674-4a05-9d92-a5063b9fa490 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:03.103422 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:03.146065 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31631859-1674-4a05-9d92-a5063b9fa490 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:03.146704 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f56bebe-6d86-4fa4-92b0-33715fb8381a None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:03.146931 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f56bebe-6d86-4fa4-92b0-33715fb8381a None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:03.147203 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f56bebe-6d86-4fa4-92b0-33715fb8381a None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:03.148535 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 483/941] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:04:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:04:03.153867 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51a4985-0e7b-4e23-ba55-eb43ab4e4d5b None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:03.154150 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51a4985-0e7b-4e23-ba55-eb43ab4e4d5b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:03.154427 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51a4985-0e7b-4e23-ba55-eb43ab4e4d5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:03.173087 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-d51a4985-0e7b-4e23-ba55-eb43ab4e4d5b None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:04:03.174337 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d51a4985-0e7b-4e23-ba55-eb43ab4e4d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:04:03.183364 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9db398-ea2b-420c-849f-4d8be1dd6cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:03.183975 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9db398-ea2b-420c-849f-4d8be1dd6cf2 None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:03.184250 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9db398-ea2b-420c-849f-4d8be1dd6cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:03.184564 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9db398-ea2b-420c-849f-4d8be1dd6cf2 None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:03.185237 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9db398-ea2b-420c-849f-4d8be1dd6cf2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:03.186680 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9db398-ea2b-420c-849f-4d8be1dd6cf2 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:03.189361 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: WARNING keystone.server.flask.common [None req-9a9db398-ea2b-420c-849f-4d8be1dd6cf2 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 06 16:04:03.210824 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.sql.core [None req-9a9db398-ea2b-420c-849f-4d8be1dd6cf2 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 06 16:04:03.210824 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 06 16:04:03.210824 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [parameters: {'id': 'caffd857905546b694395ab20ef7c21e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 06 16:04:03.210824 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81698) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 06 16:04:03.218525 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-9a9db398-ea2b-420c-849f-4d8be1dd6cf2 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id caffd857905546b694395ab20ef7c21e.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id caffd857905546b694395ab20ef7c21e. Sep 06 16:04:03.218525 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:03.218525 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 06 16:04:03.218525 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 06 16:04:03.218525 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 06 16:04:03.218525 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 06 16:04:03.218525 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 06 16:04:03.218525 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application result = self._query(query) Sep 06 16:04:03.218525 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 06 16:04:03.218525 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application conn.query(q) Sep 06 16:04:03.218525 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 06 16:04:03.218525 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 06 16:04:03.218525 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 06 16:04:03.219389 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application result.read() Sep 06 16:04:03.219389 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 06 16:04:03.219389 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 06 16:04:03.219389 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 06 16:04:03.219389 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 06 16:04:03.219389 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 06 16:04:03.219389 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 06 16:04:03.219389 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 06 16:04:03.219389 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 06 16:04:03.219389 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 06 16:04:03.219389 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:04:03.219389 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 06 16:04:03.219389 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:04:03.219389 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:03.219389 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 06 16:04:03.219389 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 06 16:04:03.219389 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 06 16:04:03.219389 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 06 16:04:03.220386 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 06 16:04:03.220386 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application next(self.gen) Sep 06 16:04:03.220386 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 06 16:04:03.220386 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application yield resource Sep 06 16:04:03.220386 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 06 16:04:03.220386 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application next(self.gen) Sep 06 16:04:03.220386 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 06 16:04:03.220386 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self.session.rollback() Sep 06 16:04:03.220386 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 06 16:04:03.220386 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self.force_reraise() Sep 06 16:04:03.220386 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 06 16:04:03.220386 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise self.value Sep 06 16:04:03.220386 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 06 16:04:03.220386 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 06 16:04:03.220386 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 06 16:04:03.220386 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application session.commit() Sep 06 16:04:03.220386 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 06 16:04:03.221387 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 06 16:04:03.221387 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 06 16:04:03.221387 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._prepare_impl() Sep 06 16:04:03.221387 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 06 16:04:03.221387 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self.session.flush() Sep 06 16:04:03.221387 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 06 16:04:03.221387 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._flush(objects) Sep 06 16:04:03.221387 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 06 16:04:03.221387 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 06 16:04:03.221387 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 06 16:04:03.221387 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application compat.raise_( Sep 06 16:04:03.221387 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 06 16:04:03.221387 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception Sep 06 16:04:03.221387 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 06 16:04:03.221387 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application flush_context.execute() Sep 06 16:04:03.221387 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 06 16:04:03.221387 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rec.execute(self) Sep 06 16:04:03.222341 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 06 16:04:03.222341 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 06 16:04:03.222341 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 06 16:04:03.222341 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 06 16:04:03.222341 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 06 16:04:03.222341 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 06 16:04:03.222341 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 06 16:04:03.222341 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 06 16:04:03.222341 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 06 16:04:03.222341 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 06 16:04:03.222341 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 06 16:04:03.222341 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 06 16:04:03.222341 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 06 16:04:03.222341 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 06 16:04:03.222341 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 06 16:04:03.223139 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 06 16:04:03.223139 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 06 16:04:03.223139 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception Sep 06 16:04:03.223139 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 06 16:04:03.223139 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 06 16:04:03.223139 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 06 16:04:03.223139 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 06 16:04:03.223139 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 06 16:04:03.223139 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application result = self._query(query) Sep 06 16:04:03.223139 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 06 16:04:03.223139 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application conn.query(q) Sep 06 16:04:03.223139 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 06 16:04:03.223139 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 06 16:04:03.223139 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 06 16:04:03.223139 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application result.read() Sep 06 16:04:03.223139 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 06 16:04:03.223139 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 06 16:04:03.225266 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 06 16:04:03.225266 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 06 16:04:03.225266 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 06 16:04:03.225266 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 06 16:04:03.225266 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 06 16:04:03.225266 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 06 16:04:03.225266 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 06 16:04:03.225266 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 06 16:04:03.225266 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [parameters: {'id': 'caffd857905546b694395ab20ef7c21e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 06 16:04:03.225266 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 06 16:04:03.225266 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:04:03.225266 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 16:04:03.225266 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:04:03.225266 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:03.225266 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 06 16:04:03.226831 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 06 16:04:03.226831 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 06 16:04:03.226831 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 06 16:04:03.226831 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 06 16:04:03.226831 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:04:03.226831 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 16:04:03.226831 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:04:03.226831 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:03.226831 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:03.226831 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:03.226831 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:03.226831 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:03.226831 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:03.226831 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:03.226831 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:03.226831 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:03.226831 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:03.228812 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:03.228812 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 06 16:04:03.228812 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 06 16:04:03.228812 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:03.228812 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:03.228812 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 06 16:04:03.228812 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 06 16:04:03.228812 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id caffd857905546b694395ab20ef7c21e. Sep 06 16:04:03.228812 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:04:03.228812 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 459/942] 10.222.0.43 () {60 vars in 1140 bytes} [Tue Sep 6 16:04:03 2022] POST /identity/v3/projects => generated 262 bytes in 186 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:04:03.228812 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9db398-ea2b-420c-849f-4d8be1dd6cf2 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:03.293391 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 484/943] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:04:03 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:04:03.349634 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9db398-ea2b-420c-849f-4d8be1dd6cf2 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:03.373024 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:03.390415 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261446f2-622e-4814-b810-a408dc5566b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:03.391109 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261446f2-622e-4814-b810-a408dc5566b9 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:03.391479 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261446f2-622e-4814-b810-a408dc5566b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:03.391846 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261446f2-622e-4814-b810-a408dc5566b9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:03.399381 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261446f2-622e-4814-b810-a408dc5566b9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:03.401307 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261446f2-622e-4814-b810-a408dc5566b9 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:03.413193 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-261446f2-622e-4814-b810-a408dc5566b9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:04:03.413193 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:03.413193 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:03.413193 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:03.413193 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:03.413193 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:03.413193 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:03.413193 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:03.413193 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:03.413193 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:03.413193 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:03.413193 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:03.413193 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:04:03.413193 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:04:03.413193 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:04:03.414236 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:04:03.414236 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:03.414236 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:03.414236 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:03.414236 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:03.414236 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:03.414236 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:03.414236 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:03.414236 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:03.414236 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:03.414236 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:03.414236 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:03.414236 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:03.414236 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:03.414236 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:03.414236 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:03.414236 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:03.415341 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:04:03.415341 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:04:03.415341 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:04:03.415341 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:04:03.415341 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:04:03.415341 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:04:03.415341 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:04:03.415341 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:04:03.416998 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 460/944] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 6 16:04:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:04:03.423022 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51a4985-0e7b-4e23-ba55-eb43ab4e4d5b None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:03.483146 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:03.529568 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51a4985-0e7b-4e23-ba55-eb43ab4e4d5b None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:03.562071 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261446f2-622e-4814-b810-a408dc5566b9 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:03.565889 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee13c90b-cdf9-4bbe-b566-9a69e285fd4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:03.566514 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee13c90b-cdf9-4bbe-b566-9a69e285fd4d None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:03.566792 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee13c90b-cdf9-4bbe-b566-9a69e285fd4d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:03.567062 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee13c90b-cdf9-4bbe-b566-9a69e285fd4d None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:03.567731 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee13c90b-cdf9-4bbe-b566-9a69e285fd4d None admin] RBAC: Adding query filter params (name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:03.568034 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee13c90b-cdf9-4bbe-b566-9a69e285fd4d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:03.569497 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee13c90b-cdf9-4bbe-b566-9a69e285fd4d None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:03.588636 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 485/945] 10.222.0.43 () {56 vars in 1112 bytes} [Tue Sep 6 16:04:03 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:03.680429 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261446f2-622e-4814-b810-a408dc5566b9 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:03.718955 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c41a07f-36dc-4a82-8234-d40b9c627de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:03.719932 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41a07f-36dc-4a82-8234-d40b9c627de7 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:03.720346 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41a07f-36dc-4a82-8234-d40b9c627de7 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:03.720860 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41a07f-36dc-4a82-8234-d40b9c627de7 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:03.721597 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c41a07f-36dc-4a82-8234-d40b9c627de7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:03.723282 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c41a07f-36dc-4a82-8234-d40b9c627de7 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:03.733869 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-2c41a07f-36dc-4a82-8234-d40b9c627de7 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 16:04:03.733869 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:03.733869 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 16:04:03.733869 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 16:04:03.733869 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 16:04:03.733869 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 16:04:03.733869 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 16:04:03.733869 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:04:03.733869 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 16:04:03.733869 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:04:03.733869 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 16:04:03.733869 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:04:03.733869 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:03.733869 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:03.733869 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:03.733869 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:03.735091 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:03.735091 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:03.735091 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:03.735091 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:03.735091 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:03.735091 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:03.735091 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:03.735091 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 16:04:03.735091 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 16:04:03.735091 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 16:04:03.735091 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 16:04:03.735091 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:03.735091 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:03.735091 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:03.735091 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:03.735091 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:03.735091 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:03.736168 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:03.736168 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:03.736168 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:03.736168 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:03.736168 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:03.736168 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:03.736168 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:03.736168 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:03.736168 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:03.736168 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:03.736168 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 16:04:03.736168 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:04:03.736168 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 16:04:03.736168 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:04:03.738753 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 461/946] 10.222.0.43 () {56 vars in 1132 bytes} [Tue Sep 6 16:04:03 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:04:03.744929 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee13c90b-cdf9-4bbe-b566-9a69e285fd4d None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:03.856227 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee13c90b-cdf9-4bbe-b566-9a69e285fd4d None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:03.870152 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:03.891287 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9d73f4-ffcd-48d6-93a1-7e3227bce388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:03.891992 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d73f4-ffcd-48d6-93a1-7e3227bce388 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:03.892327 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d73f4-ffcd-48d6-93a1-7e3227bce388 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:03.892795 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d73f4-ffcd-48d6-93a1-7e3227bce388 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:03.893516 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9d73f4-ffcd-48d6-93a1-7e3227bce388 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:03.893881 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9d73f4-ffcd-48d6-93a1-7e3227bce388 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:03.895309 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9d73f4-ffcd-48d6-93a1-7e3227bce388 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:03.900560 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 486/947] 10.222.0.43 () {56 vars in 1126 bytes} [Tue Sep 6 16:04:03 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:03.906095 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c41a07f-36dc-4a82-8234-d40b9c627de7 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:03.952470 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:04.008139 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c41a07f-36dc-4a82-8234-d40b9c627de7 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:04.042394 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ae9867-c1c3-43f2-97b5-fb5d9deba2ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:04.043001 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ae9867-c1c3-43f2-97b5-fb5d9deba2ae None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:04.043276 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ae9867-c1c3-43f2-97b5-fb5d9deba2ae None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:04.043542 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ae9867-c1c3-43f2-97b5-fb5d9deba2ae None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:04.044174 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ae9867-c1c3-43f2-97b5-fb5d9deba2ae None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:04.044467 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ae9867-c1c3-43f2-97b5-fb5d9deba2ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:04.045873 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ae9867-c1c3-43f2-97b5-fb5d9deba2ae None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:04.054075 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 462/948] 10.222.0.43 () {56 vars in 1126 bytes} [Tue Sep 6 16:04:03 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:04.059750 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9d73f4-ffcd-48d6-93a1-7e3227bce388 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:04.162245 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9d73f4-ffcd-48d6-93a1-7e3227bce388 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:04.195839 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0668f0-ae54-4add-948c-64a02bbbe398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:04.196447 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0668f0-ae54-4add-948c-64a02bbbe398 None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:04.196720 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0668f0-ae54-4add-948c-64a02bbbe398 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:04.197016 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0668f0-ae54-4add-948c-64a02bbbe398 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:04.197692 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0668f0-ae54-4add-948c-64a02bbbe398 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:04.199063 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0668f0-ae54-4add-948c-64a02bbbe398 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:04.227412 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 487/949] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:04:04 2022] POST /identity/v3/endpoints => generated 347 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:04:04.685497 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61de53d-c271-4711-a413-f90c2cda78da None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:04.686053 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61de53d-c271-4711-a413-f90c2cda78da None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:04.686053 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61de53d-c271-4711-a413-f90c2cda78da None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:04.687901 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 463/950] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:04:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:04:04.694123 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6825547d-9182-4575-b494-a5eec011ffde None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:04.694607 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6825547d-9182-4575-b494-a5eec011ffde None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:04.694716 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6825547d-9182-4575-b494-a5eec011ffde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:04.713336 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-6825547d-9182-4575-b494-a5eec011ffde None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:04:04.714508 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-6825547d-9182-4575-b494-a5eec011ffde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:04:04.741239 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:04.792105 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 488/951] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:04:04 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:04:04.940336 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2300374-3ed6-48b5-af6c-186c54bcadd9 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:04.940534 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2300374-3ed6-48b5-af6c-186c54bcadd9 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:04.940827 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2300374-3ed6-48b5-af6c-186c54bcadd9 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:04.942335 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 464/952] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:04:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:04:04.947715 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a2b092-d875-4b83-89e2-fd8991432bbb None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:04.948215 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a2b092-d875-4b83-89e2-fd8991432bbb None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:04.948702 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a2b092-d875-4b83-89e2-fd8991432bbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:04.967675 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-e8a2b092-d875-4b83-89e2-fd8991432bbb None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:04:04.969130 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e8a2b092-d875-4b83-89e2-fd8991432bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:04:05.040876 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 489/953] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:04:04 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:04:05.079738 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2300374-3ed6-48b5-af6c-186c54bcadd9 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:05.184552 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2300374-3ed6-48b5-af6c-186c54bcadd9 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:05.194172 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:05.219884 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa18282-ea38-4d16-8ba8-a5794dbf2e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:05.220607 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa18282-ea38-4d16-8ba8-a5794dbf2e1c None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:05.220874 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa18282-ea38-4d16-8ba8-a5794dbf2e1c None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:05.221147 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa18282-ea38-4d16-8ba8-a5794dbf2e1c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:05.231990 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa18282-ea38-4d16-8ba8-a5794dbf2e1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:04:05.236781 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-0aa18282-ea38-4d16-8ba8-a5794dbf2e1c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 16:04:05.236781 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:05.236781 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:05.236781 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:05.236781 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:05.236781 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:05.236781 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:05.236781 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:05.236781 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:05.236781 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:05.236781 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:05.236781 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:05.236781 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:04:05.236781 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:04:05.236781 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:04:05.236781 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:04:05.237713 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:04:05.237713 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:04:05.237713 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:04:05.237713 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:04:05.237713 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:05.237713 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:05.237713 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:05.237713 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:05.237713 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:05.237713 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:05.237713 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:05.237713 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:05.237713 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:05.237713 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:05.237713 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:05.237713 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:05.238697 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:05.238697 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:05.238697 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:05.238697 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:05.238697 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:04:05.238697 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:04:05.238697 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:04:05.238697 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:04:05.238697 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:04:05.238697 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:04:05.238697 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 16:04:05.238697 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:04:05.238697 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 465/954] 10.222.0.43 () {56 vars in 1120 bytes} [Tue Sep 6 16:04:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:04:05.249372 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a2b092-d875-4b83-89e2-fd8991432bbb None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:05.308022 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa18282-ea38-4d16-8ba8-a5794dbf2e1c None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:05.350796 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:05.361422 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a2b092-d875-4b83-89e2-fd8991432bbb None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:05.396529 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d218643-f556-4db2-a237-ba9a54135d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:05.397462 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d218643-f556-4db2-a237-ba9a54135d65 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:05.397888 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d218643-f556-4db2-a237-ba9a54135d65 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:05.398300 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d218643-f556-4db2-a237-ba9a54135d65 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:05.399112 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d218643-f556-4db2-a237-ba9a54135d65 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:05.399554 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d218643-f556-4db2-a237-ba9a54135d65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:05.401124 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d218643-f556-4db2-a237-ba9a54135d65 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:05.416331 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 490/955] 10.222.0.43 () {56 vars in 1115 bytes} [Tue Sep 6 16:04:05 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:05.417036 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa18282-ea38-4d16-8ba8-a5794dbf2e1c None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:05.417595 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d218643-f556-4db2-a237-ba9a54135d65 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:05.460885 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c52d9d1-dabe-4f7f-846f-3af567b03068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:05.461537 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c52d9d1-dabe-4f7f-846f-3af567b03068 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:05.461910 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c52d9d1-dabe-4f7f-846f-3af567b03068 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:05.462281 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c52d9d1-dabe-4f7f-846f-3af567b03068 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:05.463068 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c52d9d1-dabe-4f7f-846f-3af567b03068 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:05.464576 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c52d9d1-dabe-4f7f-846f-3af567b03068 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:05.474039 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-1c52d9d1-dabe-4f7f-846f-3af567b03068 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 16:04:05.474039 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:05.474039 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 16:04:05.474039 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 16:04:05.474039 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 16:04:05.474039 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 16:04:05.474039 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 16:04:05.474039 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:04:05.474039 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 16:04:05.474039 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:04:05.474039 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 16:04:05.474039 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:04:05.474039 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:05.474039 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:05.474039 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:05.474039 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:05.474039 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:05.475480 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:05.475480 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:05.475480 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:05.475480 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:05.475480 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:05.475480 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:05.475480 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 16:04:05.475480 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 16:04:05.475480 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 16:04:05.475480 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 16:04:05.475480 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:05.475480 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:05.475480 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:05.475480 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:05.475480 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:05.475480 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:05.476439 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:05.476439 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:05.476439 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:05.476439 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:05.476439 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:05.476439 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:05.476439 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:05.476439 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:05.476439 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:05.476439 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:05.476439 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 16:04:05.476439 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:04:05.476439 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 16:04:05.476439 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:04:05.481627 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 466/956] 10.222.0.43 () {56 vars in 1126 bytes} [Tue Sep 6 16:04:05 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:04:05.483179 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c52d9d1-dabe-4f7f-846f-3af567b03068 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:05.524921 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d218643-f556-4db2-a237-ba9a54135d65 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:05.549152 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:05.559681 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fdef9d-38f8-4236-8aa9-cb7eecd52652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:05.560598 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fdef9d-38f8-4236-8aa9-cb7eecd52652 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:05.561107 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fdef9d-38f8-4236-8aa9-cb7eecd52652 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:05.561554 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fdef9d-38f8-4236-8aa9-cb7eecd52652 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:05.573002 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fdef9d-38f8-4236-8aa9-cb7eecd52652 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:05.574778 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fdef9d-38f8-4236-8aa9-cb7eecd52652 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:05.590168 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c52d9d1-dabe-4f7f-846f-3af567b03068 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:05.590878 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-61fdef9d-38f8-4236-8aa9-cb7eecd52652 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:04:05.590878 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:05.590878 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:05.590878 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:05.590878 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:05.590878 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:05.590878 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:05.590878 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:05.590878 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:05.590878 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:05.590878 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:05.590878 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:05.590878 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:04:05.590878 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:04:05.590878 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:04:05.590878 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:04:05.591779 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:05.591779 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:05.591779 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:05.591779 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:05.591779 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:05.591779 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:05.591779 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:05.591779 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:05.591779 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:05.591779 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:05.591779 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:05.591779 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:05.591779 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:05.591779 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:05.591779 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:05.591779 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:05.591779 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:04:05.592799 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:04:05.592799 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:04:05.592799 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:04:05.599747 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 491/957] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:04:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:04:05.600973 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fdef9d-38f8-4236-8aa9-cb7eecd52652 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:05.605240 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:05.627707 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370b5f26-d702-4795-8fbc-193c78897523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:05.628565 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370b5f26-d702-4795-8fbc-193c78897523 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:05.629014 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370b5f26-d702-4795-8fbc-193c78897523 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:05.629452 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370b5f26-d702-4795-8fbc-193c78897523 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:05.630244 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370b5f26-d702-4795-8fbc-193c78897523 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:05.630731 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370b5f26-d702-4795-8fbc-193c78897523 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:05.632288 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370b5f26-d702-4795-8fbc-193c78897523 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:05.642156 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 467/958] 10.222.0.43 () {56 vars in 1122 bytes} [Tue Sep 6 16:04:05 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:04:05.643715 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370b5f26-d702-4795-8fbc-193c78897523 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:05.712209 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fdef9d-38f8-4236-8aa9-cb7eecd52652 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:05.745062 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:05.751342 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f52def4-7c41-43e4-9205-897db12068e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:05.751971 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f52def4-7c41-43e4-9205-897db12068e4 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:05.752242 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f52def4-7c41-43e4-9205-897db12068e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:05.752507 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f52def4-7c41-43e4-9205-897db12068e4 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:05.753383 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f52def4-7c41-43e4-9205-897db12068e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:05.753700 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f52def4-7c41-43e4-9205-897db12068e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:05.754960 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370b5f26-d702-4795-8fbc-193c78897523 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:05.755133 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f52def4-7c41-43e4-9205-897db12068e4 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:05.770223 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 492/959] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:04:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:05.771528 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2936783-8972-46b9-b895-190b3a11839b None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:05.771819 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2936783-8972-46b9-b895-190b3a11839b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:05.772097 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2936783-8972-46b9-b895-190b3a11839b None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:05.773482 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 493/960] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:04:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:04:05.777765 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2936783-8972-46b9-b895-190b3a11839b None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:05.801040 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52229a4e-e7fe-4806-9c18-41e6d476d3fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:05.801688 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52229a4e-e7fe-4806-9c18-41e6d476d3fa None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:05.801939 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52229a4e-e7fe-4806-9c18-41e6d476d3fa None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:05.802212 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52229a4e-e7fe-4806-9c18-41e6d476d3fa None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:05.802854 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52229a4e-e7fe-4806-9c18-41e6d476d3fa None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:05.803156 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52229a4e-e7fe-4806-9c18-41e6d476d3fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:05.804579 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52229a4e-e7fe-4806-9c18-41e6d476d3fa None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:05.809753 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 468/961] 10.222.0.43 () {56 vars in 1122 bytes} [Tue Sep 6 16:04:05 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:04:05.810955 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bac11d-bb2d-4743-a287-bee505760222 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:05.811235 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bac11d-bb2d-4743-a287-bee505760222 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:05.811504 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bac11d-bb2d-4743-a287-bee505760222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:05.831243 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-45bac11d-bb2d-4743-a287-bee505760222 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:04:05.832793 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-45bac11d-bb2d-4743-a287-bee505760222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:04:05.855588 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:05.898304 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2936783-8972-46b9-b895-190b3a11839b None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:05.913638 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 469/962] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:04:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:04:05.913638 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bac11d-bb2d-4743-a287-bee505760222 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:05.934333 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1966f57a-9e59-452e-95e1-5dcac20c01f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:05.934974 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1966f57a-9e59-452e-95e1-5dcac20c01f7 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:05.935249 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1966f57a-9e59-452e-95e1-5dcac20c01f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:05.935521 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1966f57a-9e59-452e-95e1-5dcac20c01f7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:05.944523 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1966f57a-9e59-452e-95e1-5dcac20c01f7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:05.946647 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1966f57a-9e59-452e-95e1-5dcac20c01f7 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:05.961784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-1966f57a-9e59-452e-95e1-5dcac20c01f7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 16:04:05.961784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:05.961784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:05.961784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:05.961784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:05.961784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:05.961784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:05.961784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:05.961784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:05.961784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:05.961784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:05.961784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:05.961784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:04:05.961784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:04:05.961784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:04:05.961784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:04:05.962890 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:05.962890 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:05.962890 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:04:05.962890 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:04:05.962890 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:04:05.962890 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:04:05.962890 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:05.962890 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:05.962890 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:05.962890 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:05.962890 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:05.962890 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:05.962890 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:05.962890 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:05.962890 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:05.962890 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:05.962890 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:05.962890 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:05.963946 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:05.963946 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:05.963946 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:04:05.963946 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:04:05.963946 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:04:05.963946 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:04:05.963946 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:04:05.963946 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:04:05.963946 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 16:04:05.963946 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:04:05.963946 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 494/963] 10.222.0.43 () {56 vars in 1117 bytes} [Tue Sep 6 16:04:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:04:05.969679 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1966f57a-9e59-452e-95e1-5dcac20c01f7 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:06.030221 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:06.044665 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bac11d-bb2d-4743-a287-bee505760222 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:06.091384 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415ffd2e-ac60-4f06-8a91-32c56f0795f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:06.092010 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415ffd2e-ac60-4f06-8a91-32c56f0795f0 None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:06.092290 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415ffd2e-ac60-4f06-8a91-32c56f0795f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:06.092440 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1966f57a-9e59-452e-95e1-5dcac20c01f7 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:06.092583 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415ffd2e-ac60-4f06-8a91-32c56f0795f0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:06.093253 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415ffd2e-ac60-4f06-8a91-32c56f0795f0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:06.095894 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415ffd2e-ac60-4f06-8a91-32c56f0795f0 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:06.119277 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:06.128071 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 470/964] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:04:05 2022] POST /identity/v3/endpoints => generated 349 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:04:06.143274 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b31e340-9336-4dcc-ae90-3ec895230510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:06.143908 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31e340-9336-4dcc-ae90-3ec895230510 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:06.144182 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31e340-9336-4dcc-ae90-3ec895230510 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:06.144453 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31e340-9336-4dcc-ae90-3ec895230510 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:06.145463 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b31e340-9336-4dcc-ae90-3ec895230510 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:06.145653 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b31e340-9336-4dcc-ae90-3ec895230510 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:06.147158 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b31e340-9336-4dcc-ae90-3ec895230510 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:06.169706 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 495/965] 10.222.0.43 () {56 vars in 1149 bytes} [Tue Sep 6 16:04:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:06.170026 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61907488-2a0a-46bb-8f0a-e37efdeb39c2 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:06.170551 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61907488-2a0a-46bb-8f0a-e37efdeb39c2 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:06.171069 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61907488-2a0a-46bb-8f0a-e37efdeb39c2 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:06.177228 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 471/966] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:04:06 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:04:06.179144 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61907488-2a0a-46bb-8f0a-e37efdeb39c2 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:06.179333 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2723f8c-1050-4ded-928e-2775ad5a6a61 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:06.179717 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2723f8c-1050-4ded-928e-2775ad5a6a61 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:06.180073 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2723f8c-1050-4ded-928e-2775ad5a6a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:06.201514 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-f2723f8c-1050-4ded-928e-2775ad5a6a61 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:04:06.202788 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f2723f8c-1050-4ded-928e-2775ad5a6a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:04:06.292292 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 496/967] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:04:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:04:06.293563 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2723f8c-1050-4ded-928e-2775ad5a6a61 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:06.305803 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61907488-2a0a-46bb-8f0a-e37efdeb39c2 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:06.354249 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2651f14-bf4c-4fe3-9919-f41d65b20eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:06.354863 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2651f14-bf4c-4fe3-9919-f41d65b20eb2 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:06.356386 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2651f14-bf4c-4fe3-9919-f41d65b20eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:06.356386 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2651f14-bf4c-4fe3-9919-f41d65b20eb2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:06.376768 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:06.407212 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2723f8c-1050-4ded-928e-2775ad5a6a61 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:06.417133 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:06.435422 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2651f14-bf4c-4fe3-9919-f41d65b20eb2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:06.437057 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2651f14-bf4c-4fe3-9919-f41d65b20eb2 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:06.444120 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a45e55f-4b30-4722-9e8e-549fdd0f8760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:06.444737 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a45e55f-4b30-4722-9e8e-549fdd0f8760 None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:06.445040 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a45e55f-4b30-4722-9e8e-549fdd0f8760 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:06.445309 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a45e55f-4b30-4722-9e8e-549fdd0f8760 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:06.445955 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a45e55f-4b30-4722-9e8e-549fdd0f8760 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:06.447401 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a45e55f-4b30-4722-9e8e-549fdd0f8760 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:06.450053 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: WARNING keystone.server.flask.common [None req-2a45e55f-4b30-4722-9e8e-549fdd0f8760 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 06 16:04:06.464192 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.sql.core [None req-2a45e55f-4b30-4722-9e8e-549fdd0f8760 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 06 16:04:06.464192 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 06 16:04:06.464192 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [parameters: {'user_id': 'a98390192c0c40e1bf3f756aac732873', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 06 16:04:06.464192 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81697) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 06 16:04:06.482595 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-2a45e55f-4b30-4722-9e8e-549fdd0f8760 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 06 16:04:06.482595 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:06.482595 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 06 16:04:06.482595 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 06 16:04:06.482595 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 06 16:04:06.482595 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 06 16:04:06.482595 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 06 16:04:06.482595 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application result = self._query(query) Sep 06 16:04:06.482595 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 06 16:04:06.482595 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application conn.query(q) Sep 06 16:04:06.482595 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 06 16:04:06.482595 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 06 16:04:06.482595 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 06 16:04:06.482595 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application result.read() Sep 06 16:04:06.482595 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 06 16:04:06.483632 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 06 16:04:06.483632 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 06 16:04:06.483632 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 06 16:04:06.483632 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 06 16:04:06.483632 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 06 16:04:06.483632 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 06 16:04:06.483632 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 06 16:04:06.483632 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 06 16:04:06.483632 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:04:06.483632 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 06 16:04:06.483632 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:04:06.483632 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:06.483632 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 06 16:04:06.483632 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 06 16:04:06.483632 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 06 16:04:06.483632 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 06 16:04:06.483632 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 06 16:04:06.483632 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application next(self.gen) Sep 06 16:04:06.484656 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 06 16:04:06.484656 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application yield resource Sep 06 16:04:06.484656 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 06 16:04:06.484656 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application next(self.gen) Sep 06 16:04:06.484656 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 06 16:04:06.484656 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self.session.rollback() Sep 06 16:04:06.484656 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 06 16:04:06.484656 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self.force_reraise() Sep 06 16:04:06.484656 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 06 16:04:06.484656 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise self.value Sep 06 16:04:06.484656 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 06 16:04:06.484656 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 06 16:04:06.484656 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 06 16:04:06.484656 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application session.commit() Sep 06 16:04:06.484656 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 06 16:04:06.484656 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 06 16:04:06.484656 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 06 16:04:06.487186 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._prepare_impl() Sep 06 16:04:06.487186 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 06 16:04:06.487186 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self.session.flush() Sep 06 16:04:06.487186 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 06 16:04:06.487186 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._flush(objects) Sep 06 16:04:06.487186 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 06 16:04:06.487186 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 06 16:04:06.487186 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 06 16:04:06.487186 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application compat.raise_( Sep 06 16:04:06.487186 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 06 16:04:06.487186 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception Sep 06 16:04:06.487186 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 06 16:04:06.487186 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application flush_context.execute() Sep 06 16:04:06.487186 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 06 16:04:06.487186 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rec.execute(self) Sep 06 16:04:06.487186 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 06 16:04:06.487186 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 06 16:04:06.489250 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 06 16:04:06.489250 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 06 16:04:06.489250 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 06 16:04:06.489250 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 06 16:04:06.489250 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 06 16:04:06.489250 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 06 16:04:06.489250 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 06 16:04:06.489250 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 06 16:04:06.489250 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 06 16:04:06.489250 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 06 16:04:06.489250 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 06 16:04:06.489250 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 06 16:04:06.489250 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 06 16:04:06.489250 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 06 16:04:06.489250 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 06 16:04:06.489250 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception Sep 06 16:04:06.490318 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 06 16:04:06.490318 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 06 16:04:06.490318 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 06 16:04:06.490318 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 06 16:04:06.490318 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 06 16:04:06.490318 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application result = self._query(query) Sep 06 16:04:06.490318 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 06 16:04:06.490318 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application conn.query(q) Sep 06 16:04:06.490318 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 06 16:04:06.490318 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 06 16:04:06.490318 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 06 16:04:06.490318 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application result.read() Sep 06 16:04:06.490318 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 06 16:04:06.490318 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 06 16:04:06.490318 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 06 16:04:06.490318 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 06 16:04:06.491425 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 06 16:04:06.491425 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 06 16:04:06.491425 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 06 16:04:06.491425 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 06 16:04:06.491425 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 06 16:04:06.491425 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 06 16:04:06.491425 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [parameters: {'user_id': 'a98390192c0c40e1bf3f756aac732873', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 06 16:04:06.491425 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 06 16:04:06.491425 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:04:06.491425 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 16:04:06.491425 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:04:06.491425 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:06.491425 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:06.491425 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:06.491425 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:06.492393 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:06.492393 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:06.492393 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:06.492393 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:06.492393 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:06.492393 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:06.492393 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:06.492393 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 06 16:04:06.492393 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 06 16:04:06.492393 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:06.492393 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:06.492393 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:04:06.492393 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:04:06.492393 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:04:06.492393 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:04:06.492393 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 06 16:04:06.492393 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 06 16:04:06.502191 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 06 16:04:06.502191 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 06 16:04:06.502191 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 06 16:04:06.502191 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 06 16:04:06.502191 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 06 16:04:06.502191 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:04:06.502191 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 497/968] 10.222.0.43 () {60 vars in 1131 bytes} [Tue Sep 6 16:04:06 2022] POST /identity/v3/users => generated 160 bytes in 191 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:04:06.502191 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a45e55f-4b30-4722-9e8e-549fdd0f8760 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:06.565837 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 472/969] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 6 16:04:06 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 06 16:04:06.566792 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2651f14-bf4c-4fe3-9919-f41d65b20eb2 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:06.613182 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a45e55f-4b30-4722-9e8e-549fdd0f8760 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:06.621271 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:06.650677 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c51d9eb-2af4-4e6e-9c2f-d17f870aefbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:06.651305 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c51d9eb-2af4-4e6e-9c2f-d17f870aefbb None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:06.651583 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c51d9eb-2af4-4e6e-9c2f-d17f870aefbb None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:06.651845 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c51d9eb-2af4-4e6e-9c2f-d17f870aefbb None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:06.659490 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c51d9eb-2af4-4e6e-9c2f-d17f870aefbb None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:06.661132 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c51d9eb-2af4-4e6e-9c2f-d17f870aefbb None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:06.675394 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-8c51d9eb-2af4-4e6e-9c2f-d17f870aefbb None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 06 16:04:06.675394 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:06.675394 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:06.675394 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:06.675394 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:06.675394 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:06.675394 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:06.675394 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:06.675394 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:06.675394 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:06.675394 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:06.675394 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:06.675394 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:04:06.675394 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:04:06.675394 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:04:06.675394 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:04:06.676588 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:06.676588 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:06.676588 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:04:06.676588 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:04:06.676588 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:04:06.676588 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:04:06.676588 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:06.676588 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:06.676588 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:06.676588 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:06.676588 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:06.676588 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:06.676588 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:06.676588 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:06.676588 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:06.676588 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:06.676588 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:06.676588 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:06.677965 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:06.677965 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:06.677965 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:04:06.677965 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:04:06.677965 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:04:06.677965 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:04:06.677965 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:04:06.677965 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:04:06.677965 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 06 16:04:06.677965 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:04:06.677965 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 498/970] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 6 16:04:06 2022] GET /identity/v3/users/cinder => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:04:06.677965 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c51d9eb-2af4-4e6e-9c2f-d17f870aefbb None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:06.683443 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2651f14-bf4c-4fe3-9919-f41d65b20eb2 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:06.711181 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:06.726480 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faa9ced-3c31-4d03-8197-c555790dbccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:06.727151 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa9ced-3c31-4d03-8197-c555790dbccf None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:06.727448 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa9ced-3c31-4d03-8197-c555790dbccf None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:06.727704 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa9ced-3c31-4d03-8197-c555790dbccf None admin] PATH_INFO: `/v3/services/compute` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:06.728297 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faa9ced-3c31-4d03-8197-c555790dbccf None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:06.729711 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faa9ced-3c31-4d03-8197-c555790dbccf None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:06.738620 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-4faa9ced-3c31-4d03-8197-c555790dbccf None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 16:04:06.738620 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:06.738620 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 16:04:06.738620 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 16:04:06.738620 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 16:04:06.738620 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 16:04:06.738620 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 16:04:06.738620 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:04:06.738620 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 16:04:06.738620 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:04:06.738620 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 16:04:06.738620 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:04:06.738620 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:06.738620 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:06.738620 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:06.738620 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:06.738620 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:06.744801 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:06.744801 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:06.744801 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:06.744801 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:06.744801 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:06.744801 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:06.744801 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 16:04:06.744801 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 16:04:06.744801 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 16:04:06.744801 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 16:04:06.744801 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:06.744801 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:06.744801 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:06.744801 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:06.744801 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:06.744801 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:06.746349 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:06.746349 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:06.746349 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:06.746349 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:06.746349 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:06.746349 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:06.746349 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:06.746349 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:06.746349 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:06.746349 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:06.746349 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 16:04:06.746349 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:04:06.746349 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 16:04:06.746349 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:04:06.746349 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 473/971] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 6 16:04:06 2022] GET /identity/v3/services/compute => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:04:06.747356 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faa9ced-3c31-4d03-8197-c555790dbccf None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:06.797674 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c51d9eb-2af4-4e6e-9c2f-d17f870aefbb None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:06.835005 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efc2665-36ec-48ee-a09f-7b01ded23bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:06.835615 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efc2665-36ec-48ee-a09f-7b01ded23bd2 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:06.835894 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efc2665-36ec-48ee-a09f-7b01ded23bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:06.836160 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efc2665-36ec-48ee-a09f-7b01ded23bd2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:06.846687 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faa9ced-3c31-4d03-8197-c555790dbccf None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:06.847910 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efc2665-36ec-48ee-a09f-7b01ded23bd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:06.849526 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efc2665-36ec-48ee-a09f-7b01ded23bd2 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:06.858499 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:06.864884 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-9efc2665-36ec-48ee-a09f-7b01ded23bd2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:04:06.864884 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:06.864884 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:06.864884 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:06.864884 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:06.864884 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:06.864884 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:06.864884 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:06.864884 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:06.864884 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:06.864884 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:06.864884 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:06.864884 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:04:06.864884 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:04:06.864884 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:04:06.864884 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:04:06.865845 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:06.865845 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:06.865845 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:06.865845 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:06.865845 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:06.865845 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:06.865845 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:06.865845 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:06.865845 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:06.865845 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:06.865845 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:06.865845 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:06.865845 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:06.865845 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:06.865845 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:06.865845 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:06.865845 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:04:06.867425 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:04:06.867425 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:04:06.867425 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:04:06.867425 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 499/972] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:04:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:04:06.867425 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efc2665-36ec-48ee-a09f-7b01ded23bd2 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:06.889422 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a7574-7c65-4442-b60b-a1ef3072700e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:06.890043 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a7574-7c65-4442-b60b-a1ef3072700e None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:06.890324 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a7574-7c65-4442-b60b-a1ef3072700e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:06.890644 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a7574-7c65-4442-b60b-a1ef3072700e None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:06.891355 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381a7574-7c65-4442-b60b-a1ef3072700e None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:06.891803 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381a7574-7c65-4442-b60b-a1ef3072700e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:06.893543 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381a7574-7c65-4442-b60b-a1ef3072700e None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:06.940797 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 474/973] 10.222.0.43 () {56 vars in 1101 bytes} [Tue Sep 6 16:04:06 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:06.940797 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a7574-7c65-4442-b60b-a1ef3072700e None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:06.960742 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:06.985941 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efc2665-36ec-48ee-a09f-7b01ded23bd2 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:07.027381 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbf7586-8fbc-474b-bf70-e904d46207d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:07.029013 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbf7586-8fbc-474b-bf70-e904d46207d7 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:07.029781 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbf7586-8fbc-474b-bf70-e904d46207d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:07.030261 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbf7586-8fbc-474b-bf70-e904d46207d7 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:07.031066 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbf7586-8fbc-474b-bf70-e904d46207d7 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:07.031669 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbf7586-8fbc-474b-bf70-e904d46207d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:07.034056 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbf7586-8fbc-474b-bf70-e904d46207d7 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:07.050632 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 500/974] 10.222.0.43 () {56 vars in 1112 bytes} [Tue Sep 6 16:04:06 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:07.056246 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbf7586-8fbc-474b-bf70-e904d46207d7 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:07.072427 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a7574-7c65-4442-b60b-a1ef3072700e None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:07.119523 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c92aed3-0104-48dc-b7c1-8068f9f69a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:07.120065 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c92aed3-0104-48dc-b7c1-8068f9f69a79 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:07.120337 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c92aed3-0104-48dc-b7c1-8068f9f69a79 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:07.120604 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c92aed3-0104-48dc-b7c1-8068f9f69a79 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:07.122178 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c92aed3-0104-48dc-b7c1-8068f9f69a79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:07.122178 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c92aed3-0104-48dc-b7c1-8068f9f69a79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:07.123417 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c92aed3-0104-48dc-b7c1-8068f9f69a79 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:07.137963 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 475/975] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:04:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:07.145376 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c92aed3-0104-48dc-b7c1-8068f9f69a79 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:07.182677 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:07.192105 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbf7586-8fbc-474b-bf70-e904d46207d7 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:07.229852 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7606337-3db1-4c10-a9c5-a3ea125ecca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:07.230474 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7606337-3db1-4c10-a9c5-a3ea125ecca4 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:07.230756 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7606337-3db1-4c10-a9c5-a3ea125ecca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:07.231029 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7606337-3db1-4c10-a9c5-a3ea125ecca4 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:07.231664 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7606337-3db1-4c10-a9c5-a3ea125ecca4 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:07.231966 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7606337-3db1-4c10-a9c5-a3ea125ecca4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:07.233379 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7606337-3db1-4c10-a9c5-a3ea125ecca4 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:07.237998 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 501/976] 10.222.0.43 () {56 vars in 1112 bytes} [Tue Sep 6 16:04:07 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:07.268792 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c92aed3-0104-48dc-b7c1-8068f9f69a79 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:07.309009 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df32a73-44a2-47ce-a0f2-f8eff2f7aef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:07.309697 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df32a73-44a2-47ce-a0f2-f8eff2f7aef1 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:07.309974 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df32a73-44a2-47ce-a0f2-f8eff2f7aef1 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:07.310220 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df32a73-44a2-47ce-a0f2-f8eff2f7aef1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:07.318873 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df32a73-44a2-47ce-a0f2-f8eff2f7aef1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:07.320422 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df32a73-44a2-47ce-a0f2-f8eff2f7aef1 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:07.333724 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-4df32a73-44a2-47ce-a0f2-f8eff2f7aef1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:04:07.333724 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:07.333724 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:07.333724 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:07.333724 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:07.333724 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:07.333724 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:07.333724 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:07.333724 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:07.333724 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:07.333724 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:07.333724 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:07.333724 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:04:07.333724 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:04:07.333724 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:04:07.334745 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:04:07.334745 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:07.334745 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:07.334745 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:07.334745 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:07.334745 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:07.334745 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:07.334745 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:07.334745 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:07.334745 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:07.334745 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:07.334745 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:07.334745 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:07.334745 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:07.334745 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:07.334745 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:07.334745 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:07.335666 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:04:07.335666 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:04:07.335666 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:04:07.335666 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:04:07.335666 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:04:07.335666 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:04:07.335666 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:04:07.335666 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:04:07.335666 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 476/977] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 6 16:04:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:04:07.340740 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7606337-3db1-4c10-a9c5-a3ea125ecca4 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:07.354448 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:07.449666 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7606337-3db1-4c10-a9c5-a3ea125ecca4 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:07.483498 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ad6cc2-6fec-4f82-b9ec-ae9574882112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:07.484107 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ad6cc2-6fec-4f82-b9ec-ae9574882112 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:07.484431 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ad6cc2-6fec-4f82-b9ec-ae9574882112 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:07.484702 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ad6cc2-6fec-4f82-b9ec-ae9574882112 None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:07.485411 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ad6cc2-6fec-4f82-b9ec-ae9574882112 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:07.485717 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ad6cc2-6fec-4f82-b9ec-ae9574882112 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:07.487208 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ad6cc2-6fec-4f82-b9ec-ae9574882112 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:07.500312 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 502/978] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 6 16:04:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:04:07.506301 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df32a73-44a2-47ce-a0f2-f8eff2f7aef1 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:07.600179 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:07.613610 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df32a73-44a2-47ce-a0f2-f8eff2f7aef1 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:07.651072 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b10834-d3cd-46a0-bfb1-64aadbd2d3e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:07.651650 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b10834-d3cd-46a0-bfb1-64aadbd2d3e3 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:07.651922 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b10834-d3cd-46a0-bfb1-64aadbd2d3e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:07.652190 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b10834-d3cd-46a0-bfb1-64aadbd2d3e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:07.652947 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b10834-d3cd-46a0-bfb1-64aadbd2d3e3 None admin] RBAC: Adding query filter params (role.id=012f11d05b68448696f577c7ab2161a8, scope.project.id=128d210e8dad48ac921ed4cb1d444ceb, user.id=d865006413b24ec3922619311f81f353) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:07.653241 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b10834-d3cd-46a0-bfb1-64aadbd2d3e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:07.654711 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b10834-d3cd-46a0-bfb1-64aadbd2d3e3 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:07.661010 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 477/979] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 6 16:04:07 2022] GET /identity/v3/role_assignments?user.id=d865006413b24ec3922619311f81f353&scope.project.id=128d210e8dad48ac921ed4cb1d444ceb&role.id=012f11d05b68448696f577c7ab2161a8 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:08.150641 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddb287a-6b8e-43a0-a27a-46fac039ad9f None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:08.151187 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddb287a-6b8e-43a0-a27a-46fac039ad9f None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:08.151187 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddb287a-6b8e-43a0-a27a-46fac039ad9f None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:08.152696 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 503/980] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:04:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:04:08.157832 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2bdc69-67be-40bb-a60f-516a90629357 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:08.158400 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2bdc69-67be-40bb-a60f-516a90629357 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:08.158400 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2bdc69-67be-40bb-a60f-516a90629357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:08.176719 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-3a2bdc69-67be-40bb-a60f-516a90629357 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:04:08.177933 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3a2bdc69-67be-40bb-a60f-516a90629357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:04:08.255192 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 478/981] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:04:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:04:08.521553 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddb287a-6b8e-43a0-a27a-46fac039ad9f None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:08.591060 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:08.622433 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddb287a-6b8e-43a0-a27a-46fac039ad9f None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:08.657353 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a20a628-da1b-41d5-a279-f6f8f9921817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:08.657995 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a20a628-da1b-41d5-a279-f6f8f9921817 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:08.658274 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a20a628-da1b-41d5-a279-f6f8f9921817 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:08.658552 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a20a628-da1b-41d5-a279-f6f8f9921817 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:08.659132 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a20a628-da1b-41d5-a279-f6f8f9921817 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:08.660504 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a20a628-da1b-41d5-a279-f6f8f9921817 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:08.669598 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-0a20a628-da1b-41d5-a279-f6f8f9921817 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 16:04:08.669598 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:08.669598 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 16:04:08.669598 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 16:04:08.669598 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 16:04:08.669598 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 16:04:08.669598 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 16:04:08.669598 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:04:08.669598 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 16:04:08.669598 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:04:08.669598 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 16:04:08.669598 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:04:08.669598 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:08.669598 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:08.669598 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:08.669598 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:08.669598 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:08.670619 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:08.670619 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:08.670619 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:08.670619 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:08.670619 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:08.670619 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:08.670619 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 16:04:08.670619 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 16:04:08.670619 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 16:04:08.670619 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 16:04:08.670619 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:08.670619 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:08.670619 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:08.670619 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:08.670619 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:08.670619 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:08.673902 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:08.673902 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:08.673902 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:08.673902 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:08.673902 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:08.673902 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:08.673902 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:08.673902 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:08.673902 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:08.673902 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:08.673902 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 16:04:08.673902 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:04:08.673902 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 16:04:08.673902 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:04:08.673902 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 504/982] 10.222.0.43 () {56 vars in 1126 bytes} [Tue Sep 6 16:04:08 2022] GET /identity/v3/services/object-store => generated 93 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:04:08.677279 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2bdc69-67be-40bb-a60f-516a90629357 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:08.764205 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:08.786835 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2bdc69-67be-40bb-a60f-516a90629357 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:08.820977 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025206b6-91a3-4754-adc5-c31bc5cfe478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:08.821581 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025206b6-91a3-4754-adc5-c31bc5cfe478 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:08.821866 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025206b6-91a3-4754-adc5-c31bc5cfe478 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:08.822135 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025206b6-91a3-4754-adc5-c31bc5cfe478 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:08.822801 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025206b6-91a3-4754-adc5-c31bc5cfe478 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:08.823103 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025206b6-91a3-4754-adc5-c31bc5cfe478 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:08.824488 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025206b6-91a3-4754-adc5-c31bc5cfe478 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:08.834514 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 479/983] 10.222.0.43 () {56 vars in 1122 bytes} [Tue Sep 6 16:04:08 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:04:08.836076 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a20a628-da1b-41d5-a279-f6f8f9921817 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:08.937876 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a20a628-da1b-41d5-a279-f6f8f9921817 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:08.973311 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97136a9-da16-405f-b80b-bbaf9adc0a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:08.973922 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97136a9-da16-405f-b80b-bbaf9adc0a3d None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:08.974199 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97136a9-da16-405f-b80b-bbaf9adc0a3d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:08.974471 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97136a9-da16-405f-b80b-bbaf9adc0a3d None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:08.975101 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97136a9-da16-405f-b80b-bbaf9adc0a3d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:08.975405 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97136a9-da16-405f-b80b-bbaf9adc0a3d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:08.976786 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97136a9-da16-405f-b80b-bbaf9adc0a3d None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:08.978041 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:08.982924 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 505/984] 10.222.0.43 () {56 vars in 1122 bytes} [Tue Sep 6 16:04:08 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:04:08.988880 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025206b6-91a3-4754-adc5-c31bc5cfe478 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:09.089052 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025206b6-91a3-4754-adc5-c31bc5cfe478 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:09.123090 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f13e002-d54b-4924-a207-ca3c2ab165cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:09.123720 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f13e002-d54b-4924-a207-ca3c2ab165cd None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:09.123986 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f13e002-d54b-4924-a207-ca3c2ab165cd None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:09.124257 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f13e002-d54b-4924-a207-ca3c2ab165cd None admin] PATH_INFO: `/v3/endpoints` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:09.125000 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f13e002-d54b-4924-a207-ca3c2ab165cd None admin] RBAC: Adding query filter params (interface=admin, service_id=70d749483b0d4dbcbcd119fd4379ae25, region_id=RegionOne) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:09.125307 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f13e002-d54b-4924-a207-ca3c2ab165cd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:09.126799 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f13e002-d54b-4924-a207-ca3c2ab165cd None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:09.132343 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 480/985] 10.222.0.43 () {56 vars in 1249 bytes} [Tue Sep 6 16:04:08 2022] GET /identity/v3/endpoints?service_id=70d749483b0d4dbcbcd119fd4379ae25&interface=admin®ion_id=RegionOne => generated 193 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:09.137950 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97136a9-da16-405f-b80b-bbaf9adc0a3d None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:09.138391 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350dfc68-2cdc-4860-9c69-edeffa764798 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:09.138667 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350dfc68-2cdc-4860-9c69-edeffa764798 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:09.138946 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350dfc68-2cdc-4860-9c69-edeffa764798 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:09.140962 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 481/986] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:04:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:04:09.145419 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f647e3a8-5a66-4d94-aaf0-377a32e91bba None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:09.145715 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f647e3a8-5a66-4d94-aaf0-377a32e91bba None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:09.145984 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f647e3a8-5a66-4d94-aaf0-377a32e91bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:09.165624 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-f647e3a8-5a66-4d94-aaf0-377a32e91bba None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:04:09.166802 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f647e3a8-5a66-4d94-aaf0-377a32e91bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:04:09.169687 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:09.242299 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 482/987] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:04:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:04:09.244157 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97136a9-da16-405f-b80b-bbaf9adc0a3d None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:09.285860 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462d0162-f70d-47e6-b59d-c2e937125b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:09.286690 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d0162-f70d-47e6-b59d-c2e937125b46 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:09.287129 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d0162-f70d-47e6-b59d-c2e937125b46 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:09.287537 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d0162-f70d-47e6-b59d-c2e937125b46 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:09.288298 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462d0162-f70d-47e6-b59d-c2e937125b46 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:09.289865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462d0162-f70d-47e6-b59d-c2e937125b46 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:09.296253 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 506/988] 10.222.0.43 () {56 vars in 1087 bytes} [Tue Sep 6 16:04:09 2022] GET /identity/v3/services => generated 1497 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 16:04:09.509617 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f647e3a8-5a66-4d94-aaf0-377a32e91bba None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:09.580775 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d331e3-a220-4dc2-b670-767a9abb3004 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:09.581052 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d331e3-a220-4dc2-b670-767a9abb3004 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:09.581323 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d331e3-a220-4dc2-b670-767a9abb3004 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:09.582672 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 507/989] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:04:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 16:04:09.587596 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cde429-f6c1-4557-bcd2-1e8d6923c14e None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:09.587874 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cde429-f6c1-4557-bcd2-1e8d6923c14e None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:09.588147 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cde429-f6c1-4557-bcd2-1e8d6923c14e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:09.608295 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-c5cde429-f6c1-4557-bcd2-1e8d6923c14e None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:04:09.610026 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c5cde429-f6c1-4557-bcd2-1e8d6923c14e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:04:09.614810 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f647e3a8-5a66-4d94-aaf0-377a32e91bba None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:09.649874 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c5681b-43dd-400f-92bd-979d88521d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:09.650497 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c5681b-43dd-400f-92bd-979d88521d27 None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:09.650776 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c5681b-43dd-400f-92bd-979d88521d27 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:09.651048 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c5681b-43dd-400f-92bd-979d88521d27 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:09.651724 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c5681b-43dd-400f-92bd-979d88521d27 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:09.653138 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c5681b-43dd-400f-92bd-979d88521d27 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:09.665346 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:09.674158 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 483/990] 10.222.0.43 () {60 vars in 1141 bytes} [Tue Sep 6 16:04:09 2022] POST /identity/v3/services => generated 242 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:04:09.685883 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:09.708810 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 508/991] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:04:09 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 16:04:09.973311 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c5681b-43dd-400f-92bd-979d88521d27 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:10.075421 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c5681b-43dd-400f-92bd-979d88521d27 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:10.108660 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ab82e3-3119-4025-94fa-0566e2862a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:10.109322 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab82e3-3119-4025-94fa-0566e2862a9e None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:10.109593 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab82e3-3119-4025-94fa-0566e2862a9e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:10.109856 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab82e3-3119-4025-94fa-0566e2862a9e None admin] PATH_INFO: `/v3/services/image` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:10.110441 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ab82e3-3119-4025-94fa-0566e2862a9e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:10.111902 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ab82e3-3119-4025-94fa-0566e2862a9e None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:10.120679 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-79ab82e3-3119-4025-94fa-0566e2862a9e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 16:04:10.120679 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:10.120679 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 16:04:10.120679 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 16:04:10.120679 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 16:04:10.120679 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 16:04:10.120679 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 16:04:10.120679 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:04:10.120679 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 16:04:10.120679 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:04:10.120679 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 16:04:10.120679 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:04:10.120679 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:10.120679 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:10.120679 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:10.120679 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:10.120679 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:10.121771 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:10.121771 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:10.121771 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:10.121771 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:10.121771 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:10.121771 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:10.121771 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 16:04:10.121771 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 16:04:10.121771 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 16:04:10.121771 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 16:04:10.121771 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:10.121771 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:10.121771 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:10.121771 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:10.121771 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:10.121771 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:10.122602 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:10.122602 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:10.122602 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:10.122602 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:10.122602 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:10.122602 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:10.122602 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:10.122602 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:10.122602 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:10.122602 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:10.122602 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 16:04:10.122602 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:04:10.122602 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 16:04:10.122602 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:04:10.124854 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 484/992] 10.222.0.43 () {56 vars in 1105 bytes} [Tue Sep 6 16:04:09 2022] GET /identity/v3/services/image => generated 86 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:04:10.128000 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cde429-f6c1-4557-bcd2-1e8d6923c14e None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:10.230616 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cde429-f6c1-4557-bcd2-1e8d6923c14e None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:10.270516 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdc272e-55da-423d-a6a9-2274034b5ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:10.271133 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdc272e-55da-423d-a6a9-2274034b5ca3 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:10.271410 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdc272e-55da-423d-a6a9-2274034b5ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:10.271674 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdc272e-55da-423d-a6a9-2274034b5ca3 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:10.274815 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdc272e-55da-423d-a6a9-2274034b5ca3 None admin] RBAC: Adding query filter params (name=image) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:10.274815 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdc272e-55da-423d-a6a9-2274034b5ca3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:10.274815 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdc272e-55da-423d-a6a9-2274034b5ca3 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:10.283483 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 509/993] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:04:10 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:10.286287 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ab82e3-3119-4025-94fa-0566e2862a9e None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:10.352661 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:10.395436 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ab82e3-3119-4025-94fa-0566e2862a9e None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:10.431549 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a184350-955c-4109-b4fd-ab11feee450d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:10.432154 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a184350-955c-4109-b4fd-ab11feee450d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:10.432428 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a184350-955c-4109-b4fd-ab11feee450d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:10.432697 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a184350-955c-4109-b4fd-ab11feee450d None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:10.433390 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a184350-955c-4109-b4fd-ab11feee450d None admin] RBAC: Adding query filter params (type=image) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:10.433680 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a184350-955c-4109-b4fd-ab11feee450d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:10.435064 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a184350-955c-4109-b4fd-ab11feee450d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:10.448805 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 485/994] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:04:10 2022] GET /identity/v3/services?type=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:04:11.229446 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3169fa45-555a-46ee-b513-23f39b4749d8 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:11.229728 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3169fa45-555a-46ee-b513-23f39b4749d8 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:11.230007 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3169fa45-555a-46ee-b513-23f39b4749d8 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:11.231377 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 510/995] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:04:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:04:11.236944 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb2182-0110-4b55-842c-aba1e02608f0 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:11.237287 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb2182-0110-4b55-842c-aba1e02608f0 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:11.237559 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb2182-0110-4b55-842c-aba1e02608f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:11.256912 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-5cfb2182-0110-4b55-842c-aba1e02608f0 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:04:11.258175 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5cfb2182-0110-4b55-842c-aba1e02608f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:04:11.335243 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 486/996] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:04:11 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:04:11.590402 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c41640-166f-42bd-9584-457768080331 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:11.590936 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c41640-166f-42bd-9584-457768080331 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:11.590936 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c41640-166f-42bd-9584-457768080331 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:11.592319 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 511/997] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:04:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:04:11.597623 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91323d15-720b-43a3-8455-07e242f880b3 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:11.597910 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91323d15-720b-43a3-8455-07e242f880b3 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:11.598151 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91323d15-720b-43a3-8455-07e242f880b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:11.599287 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c41640-166f-42bd-9584-457768080331 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:11.615775 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-91323d15-720b-43a3-8455-07e242f880b3 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:04:11.616986 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-91323d15-720b-43a3-8455-07e242f880b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:04:11.664436 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:11.678664 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:11.697181 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 487/998] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:04:11 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:04:11.721267 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c41640-166f-42bd-9584-457768080331 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:11.756141 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a42a0b-7f57-4fa9-bd47-7856b7db4a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:11.756788 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a42a0b-7f57-4fa9-bd47-7856b7db4a7d None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:11.757062 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a42a0b-7f57-4fa9-bd47-7856b7db4a7d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:11.757327 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a42a0b-7f57-4fa9-bd47-7856b7db4a7d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:11.757897 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a42a0b-7f57-4fa9-bd47-7856b7db4a7d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:11.759266 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a42a0b-7f57-4fa9-bd47-7856b7db4a7d None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:11.768142 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-b9a42a0b-7f57-4fa9-bd47-7856b7db4a7d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 16:04:11.768142 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:11.768142 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 16:04:11.768142 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 16:04:11.768142 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 16:04:11.768142 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 16:04:11.768142 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 16:04:11.768142 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:04:11.768142 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 16:04:11.768142 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:04:11.768142 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 16:04:11.768142 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:04:11.768142 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:11.768142 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:11.768142 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:11.768142 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:11.768142 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:11.769417 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:11.769417 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:11.769417 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:11.769417 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:11.769417 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:11.769417 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:11.769417 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 16:04:11.769417 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 16:04:11.769417 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 16:04:11.769417 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 16:04:11.769417 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:11.769417 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:11.769417 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:11.769417 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:11.769417 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:11.769417 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:11.770292 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:11.770292 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:11.770292 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:11.770292 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:11.770292 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:11.770292 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:11.770292 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:11.770292 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:11.770292 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:11.770292 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:11.770292 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 16:04:11.770292 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:04:11.770292 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 16:04:11.770292 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:04:11.773120 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 512/999] 10.222.0.43 () {56 vars in 1126 bytes} [Tue Sep 6 16:04:11 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:04:11.775399 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91323d15-720b-43a3-8455-07e242f880b3 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:11.875323 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91323d15-720b-43a3-8455-07e242f880b3 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:11.909051 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e994da1e-118e-411c-bd3f-58951d801909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:11.909664 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e994da1e-118e-411c-bd3f-58951d801909 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:11.909933 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e994da1e-118e-411c-bd3f-58951d801909 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:11.910194 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e994da1e-118e-411c-bd3f-58951d801909 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:11.910822 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e994da1e-118e-411c-bd3f-58951d801909 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:11.911116 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e994da1e-118e-411c-bd3f-58951d801909 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:11.912487 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e994da1e-118e-411c-bd3f-58951d801909 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:11.922266 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 488/1000] 10.222.0.43 () {56 vars in 1122 bytes} [Tue Sep 6 16:04:11 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:11.923530 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a42a0b-7f57-4fa9-bd47-7856b7db4a7d None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:11.958333 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e994da1e-118e-411c-bd3f-58951d801909 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:12.013517 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:12.039481 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a42a0b-7f57-4fa9-bd47-7856b7db4a7d None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:12.055714 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:12.064341 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e994da1e-118e-411c-bd3f-58951d801909 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:12.077655 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe16fde-4eb1-40b2-8f11-b1600ec3cb3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:12.078270 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe16fde-4eb1-40b2-8f11-b1600ec3cb3f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:12.078904 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe16fde-4eb1-40b2-8f11-b1600ec3cb3f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:12.079183 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe16fde-4eb1-40b2-8f11-b1600ec3cb3f None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:12.079871 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe16fde-4eb1-40b2-8f11-b1600ec3cb3f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:12.080174 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe16fde-4eb1-40b2-8f11-b1600ec3cb3f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:12.081602 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe16fde-4eb1-40b2-8f11-b1600ec3cb3f None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:12.092274 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 513/1001] 10.222.0.43 () {56 vars in 1122 bytes} [Tue Sep 6 16:04:11 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:04:12.093231 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe16fde-4eb1-40b2-8f11-b1600ec3cb3f None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:12.101713 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2614f6-4673-4889-8b46-5eb835337267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:12.102352 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2614f6-4673-4889-8b46-5eb835337267 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:12.102627 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2614f6-4673-4889-8b46-5eb835337267 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:12.102892 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2614f6-4673-4889-8b46-5eb835337267 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:12.103477 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2614f6-4673-4889-8b46-5eb835337267 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:12.104884 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2614f6-4673-4889-8b46-5eb835337267 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:12.113756 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-8d2614f6-4673-4889-8b46-5eb835337267 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 16:04:12.113756 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:12.113756 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 16:04:12.113756 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 16:04:12.113756 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 16:04:12.113756 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 16:04:12.113756 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 16:04:12.113756 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:04:12.113756 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 16:04:12.113756 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:04:12.113756 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 16:04:12.113756 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:04:12.113756 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:12.113756 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:12.113756 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:12.113756 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:12.113756 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:12.114731 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:12.114731 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:12.114731 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:12.114731 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:12.114731 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:12.114731 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:12.114731 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 16:04:12.114731 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 16:04:12.114731 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 16:04:12.114731 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 16:04:12.114731 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:12.114731 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:12.114731 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:12.114731 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:12.114731 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:12.114731 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:12.115691 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:12.115691 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:12.115691 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:12.115691 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:12.115691 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:12.115691 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:12.115691 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:12.115691 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:12.115691 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:12.115691 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:12.115691 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 16:04:12.115691 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:04:12.115691 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 16:04:12.115691 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:04:12.119847 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 489/1002] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 6 16:04:11 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:04:12.120680 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2614f6-4673-4889-8b46-5eb835337267 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:12.202764 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe16fde-4eb1-40b2-8f11-b1600ec3cb3f None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:12.226064 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2614f6-4673-4889-8b46-5eb835337267 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:12.239628 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a985dff-e4e2-4724-98df-781f88c733b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:12.240447 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a985dff-e4e2-4724-98df-781f88c733b7 None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:12.240651 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a985dff-e4e2-4724-98df-781f88c733b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:12.240944 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a985dff-e4e2-4724-98df-781f88c733b7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:12.241617 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a985dff-e4e2-4724-98df-781f88c733b7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:12.242656 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:12.244789 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a985dff-e4e2-4724-98df-781f88c733b7 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:12.264530 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 514/1003] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:04:12 2022] POST /identity/v3/endpoints => generated 325 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:04:12.283034 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e407bc34-e9e9-429c-90ef-41af1d5617d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:12.283775 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e407bc34-e9e9-429c-90ef-41af1d5617d2 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:12.284134 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e407bc34-e9e9-429c-90ef-41af1d5617d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:12.284464 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e407bc34-e9e9-429c-90ef-41af1d5617d2 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:12.285223 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e407bc34-e9e9-429c-90ef-41af1d5617d2 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:12.285588 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e407bc34-e9e9-429c-90ef-41af1d5617d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:12.287121 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e407bc34-e9e9-429c-90ef-41af1d5617d2 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:12.297857 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 490/1004] 10.222.0.43 () {56 vars in 1112 bytes} [Tue Sep 6 16:04:12 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:12.299620 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a985dff-e4e2-4724-98df-781f88c733b7 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:12.322085 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cace8c9b-5839-40c9-8160-6e028b4d9055 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:12.322414 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cace8c9b-5839-40c9-8160-6e028b4d9055 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:12.322626 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cace8c9b-5839-40c9-8160-6e028b4d9055 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:12.324197 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 491/1005] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:04:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 16:04:12.330246 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d35d6-f3f3-4f5b-9b76-5edc6069369f None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:12.330542 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d35d6-f3f3-4f5b-9b76-5edc6069369f None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:12.330837 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d35d6-f3f3-4f5b-9b76-5edc6069369f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:12.348295 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:12.349879 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-109d35d6-f3f3-4f5b-9b76-5edc6069369f None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:04:12.351532 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-109d35d6-f3f3-4f5b-9b76-5edc6069369f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:04:12.419054 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a985dff-e4e2-4724-98df-781f88c733b7 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:12.431324 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 492/1006] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:04:12 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 16:04:12.458188 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e506e82a-e024-4cf5-a1a5-33648a8c6b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:12.458809 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e506e82a-e024-4cf5-a1a5-33648a8c6b7f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:12.459223 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e506e82a-e024-4cf5-a1a5-33648a8c6b7f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:12.459554 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e506e82a-e024-4cf5-a1a5-33648a8c6b7f None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:12.460286 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e506e82a-e024-4cf5-a1a5-33648a8c6b7f None admin] RBAC: Adding query filter params (type=compute) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:12.460658 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e506e82a-e024-4cf5-a1a5-33648a8c6b7f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:12.462129 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e506e82a-e024-4cf5-a1a5-33648a8c6b7f None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:12.468258 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 515/1007] 10.222.0.43 () {56 vars in 1112 bytes} [Tue Sep 6 16:04:12 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:12.474000 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109d35d6-f3f3-4f5b-9b76-5edc6069369f None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:12.574834 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109d35d6-f3f3-4f5b-9b76-5edc6069369f None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:12.583089 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:12.610693 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158db0ed-5a29-4006-9536-5ee8bdbac0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:12.611326 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158db0ed-5a29-4006-9536-5ee8bdbac0a9 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:12.611614 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158db0ed-5a29-4006-9536-5ee8bdbac0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:12.611878 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158db0ed-5a29-4006-9536-5ee8bdbac0a9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:12.612577 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158db0ed-5a29-4006-9536-5ee8bdbac0a9 None admin] RBAC: Adding query filter params (interface=public, service_id=e04a439bd2e64520a0163da34b027506, region_id=RegionOne) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:12.612905 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158db0ed-5a29-4006-9536-5ee8bdbac0a9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:12.614283 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158db0ed-5a29-4006-9536-5ee8bdbac0a9 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:12.619743 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 493/1008] 10.222.0.43 () {56 vars in 1251 bytes} [Tue Sep 6 16:04:12 2022] GET /identity/v3/endpoints?service_id=e04a439bd2e64520a0163da34b027506&interface=public®ion_id=RegionOne => generated 194 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:04:12.625324 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e506e82a-e024-4cf5-a1a5-33648a8c6b7f None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:12.709731 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158db0ed-5a29-4006-9536-5ee8bdbac0a9 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:12.724372 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e506e82a-e024-4cf5-a1a5-33648a8c6b7f None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:12.727180 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:12.763149 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268d418d-742f-4705-a15c-d875f2113201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:12.763759 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268d418d-742f-4705-a15c-d875f2113201 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:12.764035 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268d418d-742f-4705-a15c-d875f2113201 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:12.764303 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268d418d-742f-4705-a15c-d875f2113201 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:12.764942 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268d418d-742f-4705-a15c-d875f2113201 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:12.766316 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268d418d-742f-4705-a15c-d875f2113201 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:12.781313 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 516/1009] 10.222.0.43 () {56 vars in 1087 bytes} [Tue Sep 6 16:04:12 2022] GET /identity/v3/services => generated 1727 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:04:12.826653 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158db0ed-5a29-4006-9536-5ee8bdbac0a9 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:12.868147 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc63c7cc-c329-4984-ad10-05fba9c55519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:12.869097 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc63c7cc-c329-4984-ad10-05fba9c55519 None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:12.869553 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc63c7cc-c329-4984-ad10-05fba9c55519 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:12.869967 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc63c7cc-c329-4984-ad10-05fba9c55519 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:12.870788 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc63c7cc-c329-4984-ad10-05fba9c55519 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:12.872342 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc63c7cc-c329-4984-ad10-05fba9c55519 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:12.893251 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 494/1010] 10.222.0.43 () {60 vars in 1141 bytes} [Tue Sep 6 16:04:12 2022] POST /identity/v3/services => generated 242 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:04:14.224974 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3390fd-5099-495b-9387-e120d58be36b None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:14.225522 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3390fd-5099-495b-9387-e120d58be36b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:14.225522 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3390fd-5099-495b-9387-e120d58be36b None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:14.226867 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 517/1011] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:04:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:04:14.232122 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d3c640-43fb-442c-97eb-6681bc824978 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:14.232524 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d3c640-43fb-442c-97eb-6681bc824978 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:14.232668 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d3c640-43fb-442c-97eb-6681bc824978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:14.252153 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-26d3c640-43fb-442c-97eb-6681bc824978 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:04:14.253364 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-26d3c640-43fb-442c-97eb-6681bc824978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:04:14.280056 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:14.332653 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 495/1012] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:04:14 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 16:04:14.600294 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3390fd-5099-495b-9387-e120d58be36b None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:14.654630 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50897ebc-d1e3-4833-bf1e-cabef2c5e1d9 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:14.654870 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50897ebc-d1e3-4833-bf1e-cabef2c5e1d9 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:14.655132 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50897ebc-d1e3-4833-bf1e-cabef2c5e1d9 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:14.656683 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 496/1013] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:04:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 16:04:14.661654 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b8e0b-3d88-408b-b108-2371d12aa7be None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:14.661927 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b8e0b-3d88-408b-b108-2371d12aa7be None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:14.662211 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b8e0b-3d88-408b-b108-2371d12aa7be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:14.680712 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-f47b8e0b-3d88-408b-b108-2371d12aa7be None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:04:14.681934 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f47b8e0b-3d88-408b-b108-2371d12aa7be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:04:14.711424 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3390fd-5099-495b-9387-e120d58be36b None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:14.745614 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ff4a3c-6c31-4942-a916-d4bde602e673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:14.746245 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ff4a3c-6c31-4942-a916-d4bde602e673 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:14.746519 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ff4a3c-6c31-4942-a916-d4bde602e673 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:14.746784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ff4a3c-6c31-4942-a916-d4bde602e673 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:14.747871 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ff4a3c-6c31-4942-a916-d4bde602e673 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:14.749335 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ff4a3c-6c31-4942-a916-d4bde602e673 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:14.750920 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 518/1014] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:04:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:14.754364 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 497/1015] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:04:14 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 16:04:14.756556 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ff4a3c-6c31-4942-a916-d4bde602e673 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:14.798730 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:14.805343 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdf9842-d2ec-41d4-8d7e-a59f362f6268 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:14.805746 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdf9842-d2ec-41d4-8d7e-a59f362f6268 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:14.805916 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdf9842-d2ec-41d4-8d7e-a59f362f6268 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:14.807286 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 498/1016] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:04:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 16:04:14.812021 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33288e7-8ddd-4ea9-9cf6-dce239992b8b None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:14.812317 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33288e7-8ddd-4ea9-9cf6-dce239992b8b None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:14.812589 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33288e7-8ddd-4ea9-9cf6-dce239992b8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:14.831422 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-d33288e7-8ddd-4ea9-9cf6-dce239992b8b None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:04:14.832615 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d33288e7-8ddd-4ea9-9cf6-dce239992b8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:04:14.867123 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ff4a3c-6c31-4942-a916-d4bde602e673 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:14.903860 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e85892-739e-49e5-907f-2b40a1b0ef59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:14.904297 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 499/1017] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:04:14 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 16:04:14.905313 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e85892-739e-49e5-907f-2b40a1b0ef59 None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:14.905593 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e85892-739e-49e5-907f-2b40a1b0ef59 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:14.905920 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e85892-739e-49e5-907f-2b40a1b0ef59 None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:14.906581 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e85892-739e-49e5-907f-2b40a1b0ef59 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:14.908107 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e85892-739e-49e5-907f-2b40a1b0ef59 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:14.935429 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 519/1018] 10.222.0.43 () {60 vars in 1141 bytes} [Tue Sep 6 16:04:14 2022] POST /identity/v3/projects => generated 311 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:04:15.015381 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33288e7-8ddd-4ea9-9cf6-dce239992b8b None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:15.035172 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:15.126208 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33288e7-8ddd-4ea9-9cf6-dce239992b8b None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:15.161014 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5da8e0-f63a-44c7-8042-6f866fd7aa5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:15.161657 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5da8e0-f63a-44c7-8042-6f866fd7aa5e None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:15.161929 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5da8e0-f63a-44c7-8042-6f866fd7aa5e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:15.162244 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5da8e0-f63a-44c7-8042-6f866fd7aa5e None admin] PATH_INFO: `/v3/services/compute` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:15.162813 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5da8e0-f63a-44c7-8042-6f866fd7aa5e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:15.164210 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5da8e0-f63a-44c7-8042-6f866fd7aa5e None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:15.173139 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-ab5da8e0-f63a-44c7-8042-6f866fd7aa5e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 16:04:15.173139 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:15.173139 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 16:04:15.173139 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 16:04:15.173139 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 16:04:15.173139 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 16:04:15.173139 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 16:04:15.173139 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:04:15.173139 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 16:04:15.173139 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:04:15.173139 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 16:04:15.173139 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:04:15.173139 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:15.173139 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:15.173139 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:15.173139 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:15.173139 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:15.174168 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:15.174168 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:15.174168 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:15.174168 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:15.174168 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:15.174168 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:15.174168 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 16:04:15.174168 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 16:04:15.174168 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 16:04:15.174168 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 16:04:15.174168 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:15.174168 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:15.174168 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:15.174168 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:15.174168 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:15.174168 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:15.175025 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:15.175025 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:15.175025 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:15.175025 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:15.175025 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:15.175025 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:15.175025 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:15.175025 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:15.175025 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:15.175025 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:15.175025 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 16:04:15.175025 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:04:15.175025 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 16:04:15.175025 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:04:15.175025 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 500/1019] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 6 16:04:15 2022] GET /identity/v3/services/compute => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:04:15.175928 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e85892-739e-49e5-907f-2b40a1b0ef59 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:15.180798 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5da8e0-f63a-44c7-8042-6f866fd7aa5e None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:15.266566 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:15.274787 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:15.290361 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e85892-739e-49e5-907f-2b40a1b0ef59 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:15.294833 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5da8e0-f63a-44c7-8042-6f866fd7aa5e None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:15.329082 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5a1e7f-a47a-45e8-bfc4-a42e6b203d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:15.329790 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5a1e7f-a47a-45e8-bfc4-a42e6b203d48 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:15.330068 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5a1e7f-a47a-45e8-bfc4-a42e6b203d48 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:15.330339 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5a1e7f-a47a-45e8-bfc4-a42e6b203d48 None admin] PATH_INFO: `/v3/services/image` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:15.330925 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5a1e7f-a47a-45e8-bfc4-a42e6b203d48 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:15.332060 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa57b68c-3fb6-4afd-827a-4a0565e269f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:15.332300 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5a1e7f-a47a-45e8-bfc4-a42e6b203d48 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:15.332679 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57b68c-3fb6-4afd-827a-4a0565e269f9 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:15.332971 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57b68c-3fb6-4afd-827a-4a0565e269f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:15.333241 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57b68c-3fb6-4afd-827a-4a0565e269f9 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:15.333869 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa57b68c-3fb6-4afd-827a-4a0565e269f9 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:15.334269 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa57b68c-3fb6-4afd-827a-4a0565e269f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:15.335791 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa57b68c-3fb6-4afd-827a-4a0565e269f9 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:15.341226 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-3d5a1e7f-a47a-45e8-bfc4-a42e6b203d48 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 16:04:15.341226 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:15.341226 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 16:04:15.341226 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 16:04:15.341226 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 16:04:15.341226 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 16:04:15.341226 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 16:04:15.341226 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:04:15.341226 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 16:04:15.341226 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:04:15.341226 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 16:04:15.341226 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:04:15.341226 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:15.341226 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:15.341226 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:15.341226 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:15.341226 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:15.342223 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:15.342223 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:15.342223 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:15.342223 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:15.342223 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:15.342223 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:15.342223 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 16:04:15.342223 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 16:04:15.342223 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 16:04:15.342223 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 16:04:15.342223 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:15.342223 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:15.342223 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:15.342223 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:15.342223 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:15.342223 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:15.343195 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:15.343195 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:15.343195 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:15.343195 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:15.343195 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:15.343195 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:15.343195 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:15.343195 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:15.343195 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:15.343195 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:15.343195 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 16:04:15.343195 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:04:15.343195 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 16:04:15.343195 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:04:15.345103 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 501/1020] 10.222.0.43 () {56 vars in 1112 bytes} [Tue Sep 6 16:04:15 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:04:15.346498 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa57b68c-3fb6-4afd-827a-4a0565e269f9 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:15.347068 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 520/1021] 10.222.0.43 () {56 vars in 1105 bytes} [Tue Sep 6 16:04:15 2022] GET /identity/v3/services/image => generated 86 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:04:15.347896 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5a1e7f-a47a-45e8-bfc4-a42e6b203d48 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:15.452184 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5a1e7f-a47a-45e8-bfc4-a42e6b203d48 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:15.459244 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa57b68c-3fb6-4afd-827a-4a0565e269f9 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:15.487506 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f97a7d0-05b8-47ae-83a6-aaf12da48ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:15.488113 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f97a7d0-05b8-47ae-83a6-aaf12da48ecf None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:15.488381 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f97a7d0-05b8-47ae-83a6-aaf12da48ecf None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:15.488654 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f97a7d0-05b8-47ae-83a6-aaf12da48ecf None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:15.489318 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f97a7d0-05b8-47ae-83a6-aaf12da48ecf None admin] RBAC: Adding query filter params (name=image) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:15.489617 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f97a7d0-05b8-47ae-83a6-aaf12da48ecf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:15.490996 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f97a7d0-05b8-47ae-83a6-aaf12da48ecf None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:15.495831 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 521/1022] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:04:15 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:15.496297 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bfa5f0-6ef5-4128-ae8f-b7f6783cda46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:15.497025 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bfa5f0-6ef5-4128-ae8f-b7f6783cda46 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:15.497340 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bfa5f0-6ef5-4128-ae8f-b7f6783cda46 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:15.497614 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bfa5f0-6ef5-4128-ae8f-b7f6783cda46 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:15.498293 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bfa5f0-6ef5-4128-ae8f-b7f6783cda46 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:15.498593 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bfa5f0-6ef5-4128-ae8f-b7f6783cda46 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:15.500043 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bfa5f0-6ef5-4128-ae8f-b7f6783cda46 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:15.501511 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f97a7d0-05b8-47ae-83a6-aaf12da48ecf None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:15.511291 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 502/1023] 10.222.0.43 () {56 vars in 1112 bytes} [Tue Sep 6 16:04:15 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:04:15.512310 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bfa5f0-6ef5-4128-ae8f-b7f6783cda46 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:15.521779 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:15.523583 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:15.610645 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f97a7d0-05b8-47ae-83a6-aaf12da48ecf None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:15.616441 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bfa5f0-6ef5-4128-ae8f-b7f6783cda46 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:15.647108 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476f5ede-645a-4d94-8159-e4033a8e30d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:15.647722 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476f5ede-645a-4d94-8159-e4033a8e30d0 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:15.648000 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476f5ede-645a-4d94-8159-e4033a8e30d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:15.648267 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476f5ede-645a-4d94-8159-e4033a8e30d0 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:15.648924 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476f5ede-645a-4d94-8159-e4033a8e30d0 None admin] RBAC: Adding query filter params (type=image) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:15.649236 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476f5ede-645a-4d94-8159-e4033a8e30d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:15.650599 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476f5ede-645a-4d94-8159-e4033a8e30d0 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:15.651467 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47490249-b661-4e60-b1c3-ce04c7340910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:15.652079 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47490249-b661-4e60-b1c3-ce04c7340910 None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:15.652355 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47490249-b661-4e60-b1c3-ce04c7340910 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:15.652630 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47490249-b661-4e60-b1c3-ce04c7340910 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:15.653321 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47490249-b661-4e60-b1c3-ce04c7340910 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:15.654678 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47490249-b661-4e60-b1c3-ce04c7340910 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:15.655790 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 522/1024] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:04:15 2022] GET /identity/v3/services?type=image => generated 350 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:15.661447 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476f5ede-645a-4d94-8159-e4033a8e30d0 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:15.678219 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 503/1025] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:04:15 2022] POST /identity/v3/endpoints => generated 334 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:04:15.762480 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:15.784417 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476f5ede-645a-4d94-8159-e4033a8e30d0 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:15.827689 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579246a1-8260-44da-a465-4191cd856f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:15.828655 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579246a1-8260-44da-a465-4191cd856f7f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:15.828961 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579246a1-8260-44da-a465-4191cd856f7f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:15.829230 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579246a1-8260-44da-a465-4191cd856f7f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:15.829993 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579246a1-8260-44da-a465-4191cd856f7f None admin] RBAC: Adding query filter params (interface=public, service_id=0cbdefae84034b01b62d87d167484da1, region_id=RegionOne) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:15.830314 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579246a1-8260-44da-a465-4191cd856f7f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:15.831752 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579246a1-8260-44da-a465-4191cd856f7f None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:15.839566 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 523/1026] 10.222.0.43 () {56 vars in 1251 bytes} [Tue Sep 6 16:04:15 2022] GET /identity/v3/endpoints?service_id=0cbdefae84034b01b62d87d167484da1&interface=public®ion_id=RegionOne => generated 194 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:04:15.845161 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47490249-b661-4e60-b1c3-ce04c7340910 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:15.911482 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:15.956074 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47490249-b661-4e60-b1c3-ce04c7340910 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:15.990332 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a01b754-ba34-4baf-aab9-d07cdc8e66da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:15.990947 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a01b754-ba34-4baf-aab9-d07cdc8e66da None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:15.991220 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a01b754-ba34-4baf-aab9-d07cdc8e66da None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:15.991486 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a01b754-ba34-4baf-aab9-d07cdc8e66da None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:15.992102 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a01b754-ba34-4baf-aab9-d07cdc8e66da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:15.993522 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a01b754-ba34-4baf-aab9-d07cdc8e66da None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:16.000334 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 504/1027] 10.222.0.43 () {56 vars in 1087 bytes} [Tue Sep 6 16:04:15 2022] GET /identity/v3/services => generated 1957 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:04:16.869657 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75360298-b972-4a45-ab19-2aa5838a64f4 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:16.870413 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75360298-b972-4a45-ab19-2aa5838a64f4 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:16.870413 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75360298-b972-4a45-ab19-2aa5838a64f4 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:16.871572 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 524/1028] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:04:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:04:16.876881 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1981b61-0132-401d-88c3-1842fdf15410 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:16.877294 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1981b61-0132-401d-88c3-1842fdf15410 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:16.877369 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1981b61-0132-401d-88c3-1842fdf15410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:16.895686 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-e1981b61-0132-401d-88c3-1842fdf15410 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:04:16.896870 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e1981b61-0132-401d-88c3-1842fdf15410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:04:16.965804 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 505/1029] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:04:16 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:04:17.228311 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75360298-b972-4a45-ab19-2aa5838a64f4 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:17.338769 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75360298-b972-4a45-ab19-2aa5838a64f4 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:17.374424 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218f9c15-8867-4d3d-a0ab-a3fd7e9550e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:17.375050 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218f9c15-8867-4d3d-a0ab-a3fd7e9550e4 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:17.375324 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218f9c15-8867-4d3d-a0ab-a3fd7e9550e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:17.375590 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218f9c15-8867-4d3d-a0ab-a3fd7e9550e4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:17.376712 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218f9c15-8867-4d3d-a0ab-a3fd7e9550e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:17.378175 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218f9c15-8867-4d3d-a0ab-a3fd7e9550e4 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:17.379918 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 525/1030] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:04:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:17.385705 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1981b61-0132-401d-88c3-1842fdf15410 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:17.448358 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:17.493327 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1981b61-0132-401d-88c3-1842fdf15410 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:17.527649 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1820d85-b1d8-4541-be57-b428a3358c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:17.528261 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1820d85-b1d8-4541-be57-b428a3358c92 None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:17.528543 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1820d85-b1d8-4541-be57-b428a3358c92 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:17.528843 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1820d85-b1d8-4541-be57-b428a3358c92 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:17.529509 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1820d85-b1d8-4541-be57-b428a3358c92 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:17.530974 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1820d85-b1d8-4541-be57-b428a3358c92 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:17.554314 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 506/1031] 10.222.0.43 () {60 vars in 1132 bytes} [Tue Sep 6 16:04:17 2022] POST /identity/v3/users => generated 285 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:04:17.688406 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec02604-cab6-414b-ab82-5446f59bd25c None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:17.688728 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec02604-cab6-414b-ab82-5446f59bd25c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:17.689195 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec02604-cab6-414b-ab82-5446f59bd25c None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:17.690866 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 526/1032] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:04:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:04:17.696630 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b848c28-272b-4cba-ab97-271fe169e4cd None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:17.697694 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b848c28-272b-4cba-ab97-271fe169e4cd None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:17.697972 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b848c28-272b-4cba-ab97-271fe169e4cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:17.717201 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-2b848c28-272b-4cba-ab97-271fe169e4cd None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:04:17.718540 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-2b848c28-272b-4cba-ab97-271fe169e4cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:04:17.786846 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 507/1033] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:04:17 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:04:17.902992 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d135889-9e48-4bf3-8f48-96d5680af5e8 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:17.903532 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d135889-9e48-4bf3-8f48-96d5680af5e8 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:17.903532 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d135889-9e48-4bf3-8f48-96d5680af5e8 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:17.905031 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 527/1034] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:04:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:04:17.910220 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7afb895-dbce-4982-8a53-bab0e8173c3c None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:17.910654 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7afb895-dbce-4982-8a53-bab0e8173c3c None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:17.910733 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7afb895-dbce-4982-8a53-bab0e8173c3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:17.929401 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-e7afb895-dbce-4982-8a53-bab0e8173c3c None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:04:17.930578 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e7afb895-dbce-4982-8a53-bab0e8173c3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:04:17.971890 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:18.002119 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 508/1035] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:04:17 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:04:18.049543 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d135889-9e48-4bf3-8f48-96d5680af5e8 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:18.060925 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:18.153791 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d135889-9e48-4bf3-8f48-96d5680af5e8 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:18.190704 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfe6cfa-3418-4b50-b066-e8fd3bce0d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:18.191329 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfe6cfa-3418-4b50-b066-e8fd3bce0d57 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:18.191604 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfe6cfa-3418-4b50-b066-e8fd3bce0d57 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:18.191871 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfe6cfa-3418-4b50-b066-e8fd3bce0d57 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:18.202071 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfe6cfa-3418-4b50-b066-e8fd3bce0d57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:04:18.207431 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-1bfe6cfa-3418-4b50-b066-e8fd3bce0d57 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 16:04:18.207431 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:18.207431 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:18.207431 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:18.207431 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:18.207431 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:18.207431 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:18.207431 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:18.207431 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:18.207431 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:18.207431 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:18.207431 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:18.207431 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:04:18.207431 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:04:18.207431 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:04:18.207431 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:04:18.208341 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:04:18.208341 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:04:18.208341 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:04:18.208341 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:04:18.208341 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:18.208341 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:18.208341 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:18.208341 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:18.208341 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:18.208341 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:18.208341 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:18.208341 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:18.208341 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:18.208341 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:18.208341 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:18.208341 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:18.209257 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:18.209257 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:18.209257 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:18.209257 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:18.209257 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:04:18.209257 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:04:18.209257 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:04:18.209257 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:04:18.209257 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:04:18.209257 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:04:18.209257 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 16:04:18.209257 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:04:18.213093 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 528/1036] 10.222.0.43 () {56 vars in 1120 bytes} [Tue Sep 6 16:04:18 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:04:18.214592 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7afb895-dbce-4982-8a53-bab0e8173c3c None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:18.261018 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfe6cfa-3418-4b50-b066-e8fd3bce0d57 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:18.321356 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7afb895-dbce-4982-8a53-bab0e8173c3c None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:18.332594 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:18.355379 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f37fdaa-cf22-4a82-a087-859770bed227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:18.355981 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f37fdaa-cf22-4a82-a087-859770bed227 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:18.356256 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f37fdaa-cf22-4a82-a087-859770bed227 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:18.356542 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f37fdaa-cf22-4a82-a087-859770bed227 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:18.357223 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f37fdaa-cf22-4a82-a087-859770bed227 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:18.357512 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f37fdaa-cf22-4a82-a087-859770bed227 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:18.358897 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f37fdaa-cf22-4a82-a087-859770bed227 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:18.366870 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 509/1037] 10.222.0.43 () {56 vars in 1115 bytes} [Tue Sep 6 16:04:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:18.367882 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfe6cfa-3418-4b50-b066-e8fd3bce0d57 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:18.372541 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f37fdaa-cf22-4a82-a087-859770bed227 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:18.407472 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c074a53-21ee-4029-b0a9-56d47e0f8fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:18.408112 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c074a53-21ee-4029-b0a9-56d47e0f8fa8 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:18.408373 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c074a53-21ee-4029-b0a9-56d47e0f8fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:18.408473 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:18.408625 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c074a53-21ee-4029-b0a9-56d47e0f8fa8 None admin] PATH_INFO: `/v3/services/image` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:18.409230 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c074a53-21ee-4029-b0a9-56d47e0f8fa8 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:18.410788 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c074a53-21ee-4029-b0a9-56d47e0f8fa8 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:18.419893 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-6c074a53-21ee-4029-b0a9-56d47e0f8fa8 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 16:04:18.419893 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:18.419893 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 16:04:18.419893 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 16:04:18.419893 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 16:04:18.419893 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 16:04:18.419893 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 16:04:18.419893 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:04:18.419893 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 16:04:18.419893 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:04:18.419893 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 16:04:18.419893 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:04:18.419893 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:18.419893 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:18.419893 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:18.419893 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:18.419893 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:18.420948 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:18.420948 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:18.420948 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:18.420948 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:18.420948 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:18.420948 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:18.420948 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 16:04:18.420948 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 16:04:18.420948 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 16:04:18.420948 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 16:04:18.420948 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:18.420948 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:18.420948 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:18.420948 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:18.420948 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:18.420948 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:18.421839 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:18.421839 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:18.421839 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:18.421839 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:18.421839 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:18.421839 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:18.421839 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:18.421839 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:18.421839 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:18.421839 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:18.421839 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 16:04:18.421839 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:04:18.421839 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 16:04:18.421839 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:04:18.421839 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 529/1038] 10.222.0.43 () {56 vars in 1105 bytes} [Tue Sep 6 16:04:18 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:04:18.427197 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c074a53-21ee-4029-b0a9-56d47e0f8fa8 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:18.479753 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f37fdaa-cf22-4a82-a087-859770bed227 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:18.515023 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a038b3c0-e78d-4046-a697-aff372939991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:18.515657 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a038b3c0-e78d-4046-a697-aff372939991 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:18.515937 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a038b3c0-e78d-4046-a697-aff372939991 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:18.516205 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a038b3c0-e78d-4046-a697-aff372939991 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:18.527199 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a038b3c0-e78d-4046-a697-aff372939991 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:18.528798 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a038b3c0-e78d-4046-a697-aff372939991 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:18.532522 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c074a53-21ee-4029-b0a9-56d47e0f8fa8 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:18.543779 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-a038b3c0-e78d-4046-a697-aff372939991 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:04:18.543779 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:18.543779 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:18.543779 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:18.543779 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:18.543779 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:18.543779 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:18.543779 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:18.543779 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:18.543779 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:18.543779 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:18.543779 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:18.543779 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:04:18.543779 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:04:18.543779 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:04:18.543779 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:04:18.544785 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:18.544785 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:18.544785 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:18.544785 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:18.544785 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:18.544785 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:18.544785 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:18.544785 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:18.544785 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:18.544785 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:18.544785 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:18.544785 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:18.544785 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:18.544785 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:18.544785 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:18.544785 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:18.544785 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:04:18.545695 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:04:18.545695 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:04:18.545695 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:04:18.545695 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 510/1039] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:04:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:04:18.550725 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a038b3c0-e78d-4046-a697-aff372939991 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:18.565326 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:18.570826 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddac41c-8fc1-4ba6-8413-84a08ee1a04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:18.571500 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddac41c-8fc1-4ba6-8413-84a08ee1a04d None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:18.571909 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddac41c-8fc1-4ba6-8413-84a08ee1a04d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:18.572026 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddac41c-8fc1-4ba6-8413-84a08ee1a04d None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:18.572703 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddac41c-8fc1-4ba6-8413-84a08ee1a04d None admin] RBAC: Adding query filter params (name=image) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:18.573028 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddac41c-8fc1-4ba6-8413-84a08ee1a04d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:18.574396 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddac41c-8fc1-4ba6-8413-84a08ee1a04d None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:18.579770 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 530/1040] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:04:18 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:04:18.585321 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddac41c-8fc1-4ba6-8413-84a08ee1a04d None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:18.648255 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:18.661189 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a038b3c0-e78d-4046-a697-aff372939991 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:18.694216 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddac41c-8fc1-4ba6-8413-84a08ee1a04d None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:18.697614 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd840215-5b2b-48b7-87b4-6d9414f0130f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:18.698294 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd840215-5b2b-48b7-87b4-6d9414f0130f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:18.698648 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd840215-5b2b-48b7-87b4-6d9414f0130f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:18.698978 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd840215-5b2b-48b7-87b4-6d9414f0130f None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:18.699827 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd840215-5b2b-48b7-87b4-6d9414f0130f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:18.700186 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd840215-5b2b-48b7-87b4-6d9414f0130f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:18.701687 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd840215-5b2b-48b7-87b4-6d9414f0130f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:18.714300 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 511/1041] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:04:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:04:18.719538 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd840215-5b2b-48b7-87b4-6d9414f0130f None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:18.732627 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aadcce-c0c7-4e91-98da-413f78c74e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:18.733276 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aadcce-c0c7-4e91-98da-413f78c74e03 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:18.733552 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aadcce-c0c7-4e91-98da-413f78c74e03 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:18.733817 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aadcce-c0c7-4e91-98da-413f78c74e03 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:18.734486 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aadcce-c0c7-4e91-98da-413f78c74e03 None admin] RBAC: Adding query filter params (type=image) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:18.734796 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aadcce-c0c7-4e91-98da-413f78c74e03 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:18.736174 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aadcce-c0c7-4e91-98da-413f78c74e03 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:18.740940 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 531/1042] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:04:18 2022] GET /identity/v3/services?type=image => generated 350 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:04:18.746444 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aadcce-c0c7-4e91-98da-413f78c74e03 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:18.805318 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:18.838327 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd840215-5b2b-48b7-87b4-6d9414f0130f None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:18.847910 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aadcce-c0c7-4e91-98da-413f78c74e03 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:18.875398 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca46589-98b1-4733-a93e-96b89c55d4d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:18.876153 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca46589-98b1-4733-a93e-96b89c55d4d8 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:18.876495 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca46589-98b1-4733-a93e-96b89c55d4d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:18.876865 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca46589-98b1-4733-a93e-96b89c55d4d8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:18.882626 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56354b10-1260-45ab-8d0c-77d7bd994d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:18.883296 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56354b10-1260-45ab-8d0c-77d7bd994d22 None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:18.883493 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56354b10-1260-45ab-8d0c-77d7bd994d22 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:18.883755 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56354b10-1260-45ab-8d0c-77d7bd994d22 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:18.884350 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca46589-98b1-4733-a93e-96b89c55d4d8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:18.884459 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56354b10-1260-45ab-8d0c-77d7bd994d22 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:18.885817 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56354b10-1260-45ab-8d0c-77d7bd994d22 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:18.885929 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca46589-98b1-4733-a93e-96b89c55d4d8 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:18.898362 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-2ca46589-98b1-4733-a93e-96b89c55d4d8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 16:04:18.898362 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:18.898362 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:18.898362 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:18.898362 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:18.898362 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:18.898362 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:18.898362 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:18.898362 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:18.898362 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:18.898362 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:18.898362 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:18.898362 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:04:18.898362 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:04:18.898362 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:04:18.898362 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:04:18.900380 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:18.900380 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:18.900380 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:04:18.900380 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:04:18.900380 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:04:18.900380 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:04:18.900380 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:18.900380 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:18.900380 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:18.900380 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:18.900380 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:18.900380 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:18.900380 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:18.900380 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:18.900380 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:18.900380 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:18.900380 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:18.900380 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:18.901574 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:18.901574 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:18.901574 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:04:18.901574 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:04:18.901574 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:04:18.901574 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:04:18.901574 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:04:18.901574 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:04:18.901574 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 16:04:18.901574 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:04:18.901574 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 512/1043] 10.222.0.43 () {56 vars in 1093 bytes} [Tue Sep 6 16:04:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:04:18.905675 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca46589-98b1-4733-a93e-96b89c55d4d8 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:18.920778 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 532/1044] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:04:18 2022] POST /identity/v3/endpoints => generated 327 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:04:18.930042 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:19.032014 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca46589-98b1-4733-a93e-96b89c55d4d8 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:19.079849 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511977e6-099e-47e2-9587-147ee3b32633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:19.080462 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511977e6-099e-47e2-9587-147ee3b32633 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:19.080731 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511977e6-099e-47e2-9587-147ee3b32633 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:19.081033 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511977e6-099e-47e2-9587-147ee3b32633 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:19.081766 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511977e6-099e-47e2-9587-147ee3b32633 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:19.082061 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511977e6-099e-47e2-9587-147ee3b32633 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:19.083500 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511977e6-099e-47e2-9587-147ee3b32633 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:19.101932 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 513/1045] 10.222.0.43 () {56 vars in 1133 bytes} [Tue Sep 6 16:04:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:19.107920 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56354b10-1260-45ab-8d0c-77d7bd994d22 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:19.215152 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56354b10-1260-45ab-8d0c-77d7bd994d22 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:19.231783 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:19.258896 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73e20a5-f079-4270-8ebf-bb68de62288a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:19.259514 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73e20a5-f079-4270-8ebf-bb68de62288a None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:19.259793 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73e20a5-f079-4270-8ebf-bb68de62288a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:19.260066 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73e20a5-f079-4270-8ebf-bb68de62288a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:19.334256 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73e20a5-f079-4270-8ebf-bb68de62288a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:19.335751 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73e20a5-f079-4270-8ebf-bb68de62288a None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:19.440957 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 533/1046] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 6 16:04:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 16:04:19.445701 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511977e6-099e-47e2-9587-147ee3b32633 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:19.510002 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d2fda-2d1a-472a-afe9-9647277e1ff2 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:19.510340 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d2fda-2d1a-472a-afe9-9647277e1ff2 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:19.510525 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d2fda-2d1a-472a-afe9-9647277e1ff2 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:19.512044 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 534/1047] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:04:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 16:04:19.512375 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:19.517311 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae65666-cde0-4a05-b605-aca47eaf9fbd None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:19.517560 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae65666-cde0-4a05-b605-aca47eaf9fbd None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:19.517858 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae65666-cde0-4a05-b605-aca47eaf9fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:19.535650 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-2ae65666-cde0-4a05-b605-aca47eaf9fbd None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:04:19.536877 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-2ae65666-cde0-4a05-b605-aca47eaf9fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:04:19.548457 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:19.556578 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511977e6-099e-47e2-9587-147ee3b32633 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:19.593885 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c08444-e0dc-40c3-9f2f-9be7dba2d0b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:19.594528 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c08444-e0dc-40c3-9f2f-9be7dba2d0b0 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:19.594837 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c08444-e0dc-40c3-9f2f-9be7dba2d0b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:19.595109 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c08444-e0dc-40c3-9f2f-9be7dba2d0b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:19.606211 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c08444-e0dc-40c3-9f2f-9be7dba2d0b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:19.607678 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c08444-e0dc-40c3-9f2f-9be7dba2d0b0 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:19.616868 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 535/1048] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:04:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 16:04:19.625123 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-e2c08444-e0dc-40c3-9f2f-9be7dba2d0b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:04:19.625123 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:19.625123 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:19.625123 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:19.625123 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:19.625123 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:19.625123 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:19.625123 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:19.625123 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:19.625123 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:19.625123 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:19.625123 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:19.625123 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:04:19.625123 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:04:19.625123 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:04:19.625123 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:04:19.626148 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:19.626148 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:19.626148 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:19.626148 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:19.626148 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:19.626148 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:19.626148 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:19.626148 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:19.626148 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:19.626148 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:19.626148 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:19.626148 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:19.626148 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:19.626148 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:19.626148 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:19.626148 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:19.626148 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:04:19.627095 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:04:19.627095 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:04:19.627095 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:04:19.629049 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 514/1049] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:04:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:04:19.631889 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae65666-cde0-4a05-b605-aca47eaf9fbd None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:19.732103 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae65666-cde0-4a05-b605-aca47eaf9fbd None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:19.765867 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d61152e-e00c-49a2-a5a7-9830d54d21b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:19.766597 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d61152e-e00c-49a2-a5a7-9830d54d21b5 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:19.767022 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d61152e-e00c-49a2-a5a7-9830d54d21b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:19.767391 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d61152e-e00c-49a2-a5a7-9830d54d21b5 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:19.768140 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d61152e-e00c-49a2-a5a7-9830d54d21b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:19.768543 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d61152e-e00c-49a2-a5a7-9830d54d21b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:19.770040 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d61152e-e00c-49a2-a5a7-9830d54d21b5 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:19.781642 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 536/1050] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:04:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:04:19.787277 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c08444-e0dc-40c3-9f2f-9be7dba2d0b0 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:19.883013 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d61152e-e00c-49a2-a5a7-9830d54d21b5 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:19.887260 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c08444-e0dc-40c3-9f2f-9be7dba2d0b0 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:19.892711 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:19.902011 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:19.924887 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e4c106-ab6f-427b-9eed-6cdaa12f9721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:19.925533 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e4c106-ab6f-427b-9eed-6cdaa12f9721 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:19.925811 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e4c106-ab6f-427b-9eed-6cdaa12f9721 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:19.926133 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e4c106-ab6f-427b-9eed-6cdaa12f9721 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:19.933666 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e4c106-ab6f-427b-9eed-6cdaa12f9721 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:19.935233 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e4c106-ab6f-427b-9eed-6cdaa12f9721 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:19.947654 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-87e4c106-ab6f-427b-9eed-6cdaa12f9721 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:04:19.947654 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:19.947654 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:19.947654 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:19.947654 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:19.947654 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:19.947654 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:19.947654 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:19.947654 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:19.947654 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:19.947654 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:19.947654 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:19.947654 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:04:19.947654 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:04:19.947654 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:04:19.948553 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:04:19.948553 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:19.948553 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:19.948553 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:19.948553 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:19.948553 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:19.948553 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:19.948553 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:19.948553 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:19.948553 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:19.948553 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:19.948553 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:19.948553 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:19.948553 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:19.948553 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:19.948553 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:19.948553 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:19.949724 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:04:19.949724 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:04:19.949724 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:04:19.949724 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:04:19.949724 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:04:19.949724 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:04:19.949724 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:04:19.949724 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:04:19.949724 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 515/1051] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 6 16:04:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:04:19.955104 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e4c106-ab6f-427b-9eed-6cdaa12f9721 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:19.997627 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d61152e-e00c-49a2-a5a7-9830d54d21b5 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:20.034166 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b524a275-d5e4-408a-8db5-0e0fef08e1e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:20.034815 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b524a275-d5e4-408a-8db5-0e0fef08e1e7 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:20.035083 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b524a275-d5e4-408a-8db5-0e0fef08e1e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:20.035350 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b524a275-d5e4-408a-8db5-0e0fef08e1e7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:20.045732 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b524a275-d5e4-408a-8db5-0e0fef08e1e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:04:20.051050 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-b524a275-d5e4-408a-8db5-0e0fef08e1e7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:04:20.051050 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:20.051050 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:20.051050 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:20.051050 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:20.051050 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:20.051050 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:20.051050 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:20.051050 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:20.051050 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:20.051050 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:20.051050 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:20.051050 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:04:20.051050 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:04:20.051050 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:04:20.051050 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:04:20.051976 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:04:20.051976 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:04:20.051976 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:04:20.051976 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:04:20.051976 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:20.051976 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:20.051976 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:20.051976 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:20.051976 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:20.051976 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:20.051976 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:20.051976 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:20.051976 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:20.051976 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:20.051976 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:20.051976 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:20.053209 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:20.053209 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:20.053209 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:20.053209 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:20.053209 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:04:20.053209 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:04:20.053209 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:04:20.053209 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:04:20.053209 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:04:20.053209 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:04:20.053209 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:04:20.053209 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:04:20.053209 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 537/1052] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 6 16:04:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:04:20.058165 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b524a275-d5e4-408a-8db5-0e0fef08e1e7 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:20.070147 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e4c106-ab6f-427b-9eed-6cdaa12f9721 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:20.110569 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fce184-f60a-4c23-bb75-a1af086c9ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:20.111183 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fce184-f60a-4c23-bb75-a1af086c9ca6 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:20.111462 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fce184-f60a-4c23-bb75-a1af086c9ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:20.111731 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fce184-f60a-4c23-bb75-a1af086c9ca6 None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:20.112428 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fce184-f60a-4c23-bb75-a1af086c9ca6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:20.112774 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fce184-f60a-4c23-bb75-a1af086c9ca6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:20.114229 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fce184-f60a-4c23-bb75-a1af086c9ca6 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:20.126680 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 516/1053] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 6 16:04:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:04:20.128875 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:20.132701 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fce184-f60a-4c23-bb75-a1af086c9ca6 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:20.164845 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:20.169077 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b524a275-d5e4-408a-8db5-0e0fef08e1e7 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:20.206605 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e7ed29-5a80-43f5-bcd2-7f166d39bbdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:20.207218 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e7ed29-5a80-43f5-bcd2-7f166d39bbdc None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:20.207490 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e7ed29-5a80-43f5-bcd2-7f166d39bbdc None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:20.207760 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e7ed29-5a80-43f5-bcd2-7f166d39bbdc None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:20.208852 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e7ed29-5a80-43f5-bcd2-7f166d39bbdc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:20.209159 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e7ed29-5a80-43f5-bcd2-7f166d39bbdc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:20.210913 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e7ed29-5a80-43f5-bcd2-7f166d39bbdc None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:20.227190 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 538/1054] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 6 16:04:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:04:20.228903 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e7ed29-5a80-43f5-bcd2-7f166d39bbdc None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:20.233507 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fce184-f60a-4c23-bb75-a1af086c9ca6 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:20.271636 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d613ba25-ee4d-4e4e-acbe-3ba9a6f242f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:20.272347 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d613ba25-ee4d-4e4e-acbe-3ba9a6f242f8 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:20.272662 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d613ba25-ee4d-4e4e-acbe-3ba9a6f242f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:20.272945 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d613ba25-ee4d-4e4e-acbe-3ba9a6f242f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:20.273670 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d613ba25-ee4d-4e4e-acbe-3ba9a6f242f8 None admin] RBAC: Adding query filter params (role.id=012f11d05b68448696f577c7ab2161a8, scope.project.id=128d210e8dad48ac921ed4cb1d444ceb, user.id=63de41f4348243998029f7d07aa5dc40) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:20.273971 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d613ba25-ee4d-4e4e-acbe-3ba9a6f242f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:20.275464 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d613ba25-ee4d-4e4e-acbe-3ba9a6f242f8 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:20.289230 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 517/1055] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 6 16:04:20 2022] GET /identity/v3/role_assignments?user.id=63de41f4348243998029f7d07aa5dc40&scope.project.id=128d210e8dad48ac921ed4cb1d444ceb&role.id=012f11d05b68448696f577c7ab2161a8 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:04:20.350699 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e7ed29-5a80-43f5-bcd2-7f166d39bbdc None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:20.380085 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:20.390005 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f75656-1fae-4f63-8e76-4d70a33ba7b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:20.390661 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f75656-1fae-4f63-8e76-4d70a33ba7b2 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:20.390940 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f75656-1fae-4f63-8e76-4d70a33ba7b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:20.391211 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f75656-1fae-4f63-8e76-4d70a33ba7b2 None admin] PATH_INFO: `/v3/users/85cf8128739f4389ab4e26559f94877d` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:20.407106 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f75656-1fae-4f63-8e76-4d70a33ba7b2 None admin] RBAC: Authorizing `identity:get_user(user_id=85cf8128739f4389ab4e26559f94877d)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:20.408640 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f75656-1fae-4f63-8e76-4d70a33ba7b2 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:20.410616 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 539/1056] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 6 16:04:20 2022] GET /identity/v3/users/85cf8128739f4389ab4e26559f94877d => generated 285 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:04:20.416609 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d613ba25-ee4d-4e4e-acbe-3ba9a6f242f8 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:20.521051 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:20.525722 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d613ba25-ee4d-4e4e-acbe-3ba9a6f242f8 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:20.560806 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a1fbef-48d4-42d4-96f7-edd9fb7023bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:20.561444 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a1fbef-48d4-42d4-96f7-edd9fb7023bc None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:20.561718 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a1fbef-48d4-42d4-96f7-edd9fb7023bc None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:20.561984 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a1fbef-48d4-42d4-96f7-edd9fb7023bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:20.629750 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a1fbef-48d4-42d4-96f7-edd9fb7023bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:20.631213 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a1fbef-48d4-42d4-96f7-edd9fb7023bc None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:20.729003 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:20.734790 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 518/1057] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 6 16:04:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 16:04:20.740447 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f75656-1fae-4f63-8e76-4d70a33ba7b2 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:20.842287 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f75656-1fae-4f63-8e76-4d70a33ba7b2 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:20.854281 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e84f6-7e0e-4d19-b897-1659416c9a8b None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:20.854556 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e84f6-7e0e-4d19-b897-1659416c9a8b None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:20.854840 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e84f6-7e0e-4d19-b897-1659416c9a8b None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:20.856317 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 519/1058] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:04:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:04:20.861274 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2184c42-06d7-4864-bbfc-d0e14b85acff None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:20.861565 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2184c42-06d7-4864-bbfc-d0e14b85acff None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:20.861842 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2184c42-06d7-4864-bbfc-d0e14b85acff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:20.877581 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb513df-f32e-4de8-b3f9-2d6b692ea740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:20.878226 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb513df-f32e-4de8-b3f9-2d6b692ea740 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:20.878505 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb513df-f32e-4de8-b3f9-2d6b692ea740 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:20.878774 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb513df-f32e-4de8-b3f9-2d6b692ea740 None admin] PATH_INFO: `/v3/projects/3c1d8bb4e7ad45e28b1040214a1a5e1a` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:20.879864 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb513df-f32e-4de8-b3f9-2d6b692ea740 None admin] RBAC: Authorizing `identity:get_project(project_id=3c1d8bb4e7ad45e28b1040214a1a5e1a)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:20.880352 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-e2184c42-06d7-4864-bbfc-d0e14b85acff None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:04:20.881367 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb513df-f32e-4de8-b3f9-2d6b692ea740 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:20.881571 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e2184c42-06d7-4864-bbfc-d0e14b85acff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:04:20.888079 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 540/1059] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 6 16:04:20 2022] GET /identity/v3/projects/3c1d8bb4e7ad45e28b1040214a1a5e1a => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:20.889600 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb513df-f32e-4de8-b3f9-2d6b692ea740 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:20.945420 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:20.957868 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 520/1060] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:04:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 16:04:20.998064 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb513df-f32e-4de8-b3f9-2d6b692ea740 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:21.031891 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2924ba27-4fcb-40f1-a5f0-400cf0191785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:21.032576 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924ba27-4fcb-40f1-a5f0-400cf0191785 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:21.032845 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924ba27-4fcb-40f1-a5f0-400cf0191785 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:21.033137 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924ba27-4fcb-40f1-a5f0-400cf0191785 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:21.033888 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2924ba27-4fcb-40f1-a5f0-400cf0191785 None admin] RBAC: Adding query filter params (role.id=edaabdea8be84a7980fb2c651fe52f63, scope.project.id=3c1d8bb4e7ad45e28b1040214a1a5e1a, user.id=85cf8128739f4389ab4e26559f94877d) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:21.034187 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2924ba27-4fcb-40f1-a5f0-400cf0191785 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:21.035700 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2924ba27-4fcb-40f1-a5f0-400cf0191785 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:21.041439 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 541/1061] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 6 16:04:20 2022] GET /identity/v3/role_assignments?user.id=85cf8128739f4389ab4e26559f94877d&scope.project.id=3c1d8bb4e7ad45e28b1040214a1a5e1a&role.id=edaabdea8be84a7980fb2c651fe52f63 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:21.224802 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2184c42-06d7-4864-bbfc-d0e14b85acff None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:21.342316 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2184c42-06d7-4864-bbfc-d0e14b85acff None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:21.370906 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:21.379752 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc9b66a-6b19-4c31-a830-b1fe7da6c8ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:21.380369 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9b66a-6b19-4c31-a830-b1fe7da6c8ad None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:21.380646 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9b66a-6b19-4c31-a830-b1fe7da6c8ad None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:21.380949 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9b66a-6b19-4c31-a830-b1fe7da6c8ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:21.450737 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc9b66a-6b19-4c31-a830-b1fe7da6c8ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:21.452215 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc9b66a-6b19-4c31-a830-b1fe7da6c8ad None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:21.554128 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 521/1062] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 6 16:04:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 16:04:21.559859 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2924ba27-4fcb-40f1-a5f0-400cf0191785 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:21.665405 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2924ba27-4fcb-40f1-a5f0-400cf0191785 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:21.685948 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:21.704733 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dfcfef-996f-4057-9e4c-d64129960558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:21.705395 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dfcfef-996f-4057-9e4c-d64129960558 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:21.705668 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dfcfef-996f-4057-9e4c-d64129960558 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:21.705934 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dfcfef-996f-4057-9e4c-d64129960558 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:21.716233 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dfcfef-996f-4057-9e4c-d64129960558 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:21.717729 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dfcfef-996f-4057-9e4c-d64129960558 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:21.731507 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-c0dfcfef-996f-4057-9e4c-d64129960558 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:04:21.731507 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:21.731507 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:21.731507 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:21.731507 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:21.731507 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:21.731507 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:21.731507 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:21.731507 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:21.731507 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:21.731507 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:21.731507 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:21.731507 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:04:21.731507 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:04:21.731507 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:04:21.731507 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:04:21.732476 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:21.732476 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:21.732476 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:21.732476 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:21.732476 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:21.732476 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:21.732476 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:21.732476 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:21.732476 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:21.732476 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:21.732476 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:21.732476 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:21.732476 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:21.732476 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:21.732476 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:21.732476 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:21.732476 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:04:21.737772 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:04:21.737772 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:04:21.737772 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:04:21.737772 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 542/1063] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:04:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:04:21.740284 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc9b66a-6b19-4c31-a830-b1fe7da6c8ad None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:21.770614 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:21.848090 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc9b66a-6b19-4c31-a830-b1fe7da6c8ad None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:21.882903 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e8dd4e-e424-474c-8f79-5cc09dbb32ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:21.883518 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e8dd4e-e424-474c-8f79-5cc09dbb32ef None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:21.883787 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e8dd4e-e424-474c-8f79-5cc09dbb32ef None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:21.884057 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e8dd4e-e424-474c-8f79-5cc09dbb32ef None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:21.884775 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e8dd4e-e424-474c-8f79-5cc09dbb32ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:21.885093 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e8dd4e-e424-474c-8f79-5cc09dbb32ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:21.886594 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e8dd4e-e424-474c-8f79-5cc09dbb32ef None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:21.901100 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 522/1064] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:04:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:22.182585 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0d5d6-6678-46fb-be4c-dd3596b002d2 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:22.183085 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0d5d6-6678-46fb-be4c-dd3596b002d2 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:22.183085 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0d5d6-6678-46fb-be4c-dd3596b002d2 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:22.184485 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 543/1065] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:04:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:04:22.189858 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf189cd-f6b5-4775-90ec-581f94bce9ab None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:22.190352 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf189cd-f6b5-4775-90ec-581f94bce9ab None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:22.190529 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf189cd-f6b5-4775-90ec-581f94bce9ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:22.211169 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-2bf189cd-f6b5-4775-90ec-581f94bce9ab None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:04:22.212378 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-2bf189cd-f6b5-4775-90ec-581f94bce9ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:04:22.285341 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 523/1066] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:04:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 16:04:22.548259 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe0d5d6-6678-46fb-be4c-dd3596b002d2 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:22.651711 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe0d5d6-6678-46fb-be4c-dd3596b002d2 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:22.686530 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afabff95-bbbb-417b-9e8b-4ef06a82a5a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:22.687158 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afabff95-bbbb-417b-9e8b-4ef06a82a5a2 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:22.687435 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afabff95-bbbb-417b-9e8b-4ef06a82a5a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:22.687700 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afabff95-bbbb-417b-9e8b-4ef06a82a5a2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:22.698291 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afabff95-bbbb-417b-9e8b-4ef06a82a5a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:04:22.703559 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-afabff95-bbbb-417b-9e8b-4ef06a82a5a2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 16:04:22.703559 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:22.703559 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:22.703559 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:22.703559 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:22.703559 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:22.703559 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:22.703559 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:22.703559 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:22.703559 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:22.703559 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:22.703559 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:22.703559 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:04:22.703559 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:04:22.703559 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:04:22.703559 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:04:22.704631 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:04:22.704631 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:04:22.704631 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:04:22.704631 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:04:22.704631 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:22.704631 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:22.704631 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:22.704631 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:22.704631 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:22.704631 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:22.704631 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:22.704631 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:22.704631 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:22.704631 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:22.704631 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:22.704631 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:22.705502 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:22.705502 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:22.705502 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:22.705502 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:22.705502 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:04:22.705502 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:04:22.705502 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:04:22.705502 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:04:22.705502 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:04:22.705502 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:04:22.705502 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 16:04:22.705502 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:04:22.705502 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 544/1067] 10.222.0.43 () {56 vars in 1120 bytes} [Tue Sep 6 16:04:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:04:22.710924 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf189cd-f6b5-4775-90ec-581f94bce9ab None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:22.729407 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:22.811864 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf189cd-f6b5-4775-90ec-581f94bce9ab None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:22.847450 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce5e1d3-3b09-47b0-bd09-90df1c9cf8c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:22.848047 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce5e1d3-3b09-47b0-bd09-90df1c9cf8c6 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:22.848323 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce5e1d3-3b09-47b0-bd09-90df1c9cf8c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:22.848591 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce5e1d3-3b09-47b0-bd09-90df1c9cf8c6 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:22.849284 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce5e1d3-3b09-47b0-bd09-90df1c9cf8c6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:22.849582 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce5e1d3-3b09-47b0-bd09-90df1c9cf8c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:22.850975 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce5e1d3-3b09-47b0-bd09-90df1c9cf8c6 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:22.859060 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 524/1068] 10.222.0.43 () {56 vars in 1115 bytes} [Tue Sep 6 16:04:22 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:22.865164 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afabff95-bbbb-417b-9e8b-4ef06a82a5a2 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:22.897242 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:22.897550 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab9d77-2231-416b-a3a7-87fe3e82dec9 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:22.897900 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab9d77-2231-416b-a3a7-87fe3e82dec9 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:22.898193 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab9d77-2231-416b-a3a7-87fe3e82dec9 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:22.899644 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 525/1069] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:04:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 16:04:22.904587 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca6671d-68ba-4ca7-a745-a6cb292e20fc None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:22.904906 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca6671d-68ba-4ca7-a745-a6cb292e20fc None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:22.905181 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca6671d-68ba-4ca7-a745-a6cb292e20fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:22.924544 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-1ca6671d-68ba-4ca7-a745-a6cb292e20fc None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:04:22.925792 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1ca6671d-68ba-4ca7-a745-a6cb292e20fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:04:22.969607 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afabff95-bbbb-417b-9e8b-4ef06a82a5a2 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:22.990939 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 526/1070] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:04:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 16:04:23.003374 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf1be6e-3e2e-4ea0-8a0c-8cc7a68c66fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:23.004008 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf1be6e-3e2e-4ea0-8a0c-8cc7a68c66fc None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:23.004274 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf1be6e-3e2e-4ea0-8a0c-8cc7a68c66fc None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:23.004538 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf1be6e-3e2e-4ea0-8a0c-8cc7a68c66fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:23.014437 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf1be6e-3e2e-4ea0-8a0c-8cc7a68c66fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:23.015887 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf1be6e-3e2e-4ea0-8a0c-8cc7a68c66fc None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:23.029963 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-8cf1be6e-3e2e-4ea0-8a0c-8cc7a68c66fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:04:23.029963 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:23.029963 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:23.029963 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:23.029963 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:23.029963 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:23.029963 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:23.029963 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:23.029963 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:23.029963 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:23.029963 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:23.029963 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:23.029963 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:04:23.029963 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:04:23.029963 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:04:23.029963 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:04:23.030908 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:23.030908 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:23.030908 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:23.030908 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:23.030908 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:23.030908 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:23.030908 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:23.030908 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:23.030908 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:23.030908 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:23.030908 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:23.030908 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:23.030908 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:23.030908 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:23.030908 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:23.030908 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:23.030908 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:04:23.031811 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:04:23.031811 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:04:23.031811 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:04:23.031811 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 545/1071] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:04:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:04:23.036955 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca6671d-68ba-4ca7-a745-a6cb292e20fc None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:23.049159 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:23.137823 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca6671d-68ba-4ca7-a745-a6cb292e20fc None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:23.171730 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccd79c8-183b-4e59-9875-5475b9813db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:23.172326 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccd79c8-183b-4e59-9875-5475b9813db3 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:23.172593 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccd79c8-183b-4e59-9875-5475b9813db3 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:23.172920 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccd79c8-183b-4e59-9875-5475b9813db3 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:23.173559 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccd79c8-183b-4e59-9875-5475b9813db3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:23.173850 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccd79c8-183b-4e59-9875-5475b9813db3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:23.175225 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccd79c8-183b-4e59-9875-5475b9813db3 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:23.186719 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 527/1072] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:04:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:23.192582 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf1be6e-3e2e-4ea0-8a0c-8cc7a68c66fc None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:23.258522 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccd79c8-183b-4e59-9875-5475b9813db3 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:23.278073 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:23.296879 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf1be6e-3e2e-4ea0-8a0c-8cc7a68c66fc None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:23.330296 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb63ea55-1e28-4321-99fa-20b43455f381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:23.330503 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:23.330921 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb63ea55-1e28-4321-99fa-20b43455f381 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:23.331190 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb63ea55-1e28-4321-99fa-20b43455f381 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:23.331444 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb63ea55-1e28-4321-99fa-20b43455f381 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:23.339727 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb63ea55-1e28-4321-99fa-20b43455f381 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:23.341212 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb63ea55-1e28-4321-99fa-20b43455f381 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:23.352082 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-cb63ea55-1e28-4321-99fa-20b43455f381 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 16:04:23.352082 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:23.352082 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:23.352082 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:23.352082 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:23.352082 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:23.352082 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:23.352082 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:23.352082 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:23.352082 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:23.352082 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:23.352082 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:23.352082 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:04:23.352082 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:04:23.352082 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:04:23.352082 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:04:23.353076 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:23.353076 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:23.353076 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:04:23.353076 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:04:23.353076 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:04:23.353076 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:04:23.353076 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:23.353076 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:23.353076 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:23.353076 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:23.353076 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:23.353076 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:23.353076 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:23.353076 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:23.353076 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:23.353076 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:23.353076 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:23.353076 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:23.354061 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:23.354061 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:23.354061 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:04:23.354061 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:04:23.354061 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:04:23.354061 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:04:23.354061 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:04:23.354061 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:04:23.354061 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 16:04:23.354061 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:04:23.354061 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 546/1073] 10.222.0.43 () {56 vars in 1093 bytes} [Tue Sep 6 16:04:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:04:23.359038 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb63ea55-1e28-4321-99fa-20b43455f381 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:23.363988 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccd79c8-183b-4e59-9875-5475b9813db3 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:23.397734 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253a2e24-61cc-4cbb-a24e-eef5a421feac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:23.398299 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a2e24-61cc-4cbb-a24e-eef5a421feac None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:23.398578 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a2e24-61cc-4cbb-a24e-eef5a421feac None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:23.398849 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a2e24-61cc-4cbb-a24e-eef5a421feac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:23.409429 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253a2e24-61cc-4cbb-a24e-eef5a421feac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:04:23.414270 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-253a2e24-61cc-4cbb-a24e-eef5a421feac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:04:23.414270 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:23.414270 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:23.414270 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:23.414270 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:23.414270 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:23.414270 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:23.414270 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:23.414270 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:23.414270 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:23.414270 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:23.414270 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:23.414270 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:04:23.414270 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:04:23.414270 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:04:23.414270 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:04:23.415225 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:04:23.415225 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:04:23.415225 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:04:23.415225 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:04:23.415225 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:23.415225 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:23.415225 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:23.415225 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:23.415225 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:23.415225 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:23.415225 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:23.415225 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:23.415225 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:23.415225 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:23.415225 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:23.415225 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:23.416084 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:23.416084 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:23.416084 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:23.416084 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:23.416084 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:04:23.416084 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:04:23.416084 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:04:23.416084 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:04:23.416084 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:04:23.416084 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:04:23.416084 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:04:23.416084 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:04:23.416084 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 528/1074] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 6 16:04:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:04:23.421399 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253a2e24-61cc-4cbb-a24e-eef5a421feac None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:23.462544 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb63ea55-1e28-4321-99fa-20b43455f381 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:23.499600 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f36bd17-fefd-4629-af4f-c0a6f4255d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:23.500205 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f36bd17-fefd-4629-af4f-c0a6f4255d3a None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:23.500517 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f36bd17-fefd-4629-af4f-c0a6f4255d3a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:23.500782 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f36bd17-fefd-4629-af4f-c0a6f4255d3a None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:23.501566 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f36bd17-fefd-4629-af4f-c0a6f4255d3a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:23.501865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f36bd17-fefd-4629-af4f-c0a6f4255d3a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:23.503297 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f36bd17-fefd-4629-af4f-c0a6f4255d3a None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:23.517614 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:23.520961 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 547/1075] 10.222.0.43 () {56 vars in 1133 bytes} [Tue Sep 6 16:04:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:23.522765 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253a2e24-61cc-4cbb-a24e-eef5a421feac None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:23.526833 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f36bd17-fefd-4629-af4f-c0a6f4255d3a None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:23.551042 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:23.557571 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce4947a-6c8a-41a1-b419-7280508f495f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:23.558180 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce4947a-6c8a-41a1-b419-7280508f495f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:23.558455 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce4947a-6c8a-41a1-b419-7280508f495f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:23.558724 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce4947a-6c8a-41a1-b419-7280508f495f None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:23.559394 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce4947a-6c8a-41a1-b419-7280508f495f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:23.559694 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce4947a-6c8a-41a1-b419-7280508f495f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:23.561110 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce4947a-6c8a-41a1-b419-7280508f495f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:23.569729 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 529/1076] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 6 16:04:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:04:23.575523 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce4947a-6c8a-41a1-b419-7280508f495f None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:23.629058 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f36bd17-fefd-4629-af4f-c0a6f4255d3a None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:23.661785 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3898e1-86d7-4a46-8469-99d2107826a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:23.662343 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3898e1-86d7-4a46-8469-99d2107826a8 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:23.662642 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3898e1-86d7-4a46-8469-99d2107826a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:23.662903 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3898e1-86d7-4a46-8469-99d2107826a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:23.672515 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3898e1-86d7-4a46-8469-99d2107826a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:23.673155 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce4947a-6c8a-41a1-b419-7280508f495f None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:23.674041 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3898e1-86d7-4a46-8469-99d2107826a8 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:23.689265 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-fa3898e1-86d7-4a46-8469-99d2107826a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:04:23.689265 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:23.689265 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:23.689265 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:23.689265 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:23.689265 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:23.689265 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:23.689265 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:23.689265 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:23.689265 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:23.689265 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:23.689265 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:23.689265 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:04:23.689265 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:04:23.689265 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:04:23.689265 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:04:23.690304 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:23.690304 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:23.690304 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:23.690304 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:23.690304 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:23.690304 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:23.690304 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:23.690304 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:23.690304 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:23.690304 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:23.690304 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:23.690304 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:23.690304 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:23.690304 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:23.690304 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:23.690304 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:23.690304 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:04:23.691211 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:04:23.691211 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:04:23.691211 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:04:23.691211 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 548/1077] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:04:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:04:23.696297 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3898e1-86d7-4a46-8469-99d2107826a8 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:23.710417 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d8c753-27bb-4fe3-82cb-0543b5188ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:23.711046 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d8c753-27bb-4fe3-82cb-0543b5188ca7 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:23.711327 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d8c753-27bb-4fe3-82cb-0543b5188ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:23.711591 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d8c753-27bb-4fe3-82cb-0543b5188ca7 None admin] PATH_INFO: `/v3/users/85cf8128739f4389ab4e26559f94877d` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:23.713124 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d8c753-27bb-4fe3-82cb-0543b5188ca7 None admin] RBAC: Authorizing `identity:get_user(user_id=85cf8128739f4389ab4e26559f94877d)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:23.714229 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d8c753-27bb-4fe3-82cb-0543b5188ca7 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:23.715833 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 530/1078] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 6 16:04:23 2022] GET /identity/v3/users/85cf8128739f4389ab4e26559f94877d => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:23.721620 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d8c753-27bb-4fe3-82cb-0543b5188ca7 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:23.750549 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:23.780543 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:23.798218 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3898e1-86d7-4a46-8469-99d2107826a8 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:23.820882 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d8c753-27bb-4fe3-82cb-0543b5188ca7 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:23.832571 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1426489-b5fb-4b57-bfe3-94b504800901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:23.833202 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1426489-b5fb-4b57-bfe3-94b504800901 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:23.833475 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1426489-b5fb-4b57-bfe3-94b504800901 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:23.833737 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1426489-b5fb-4b57-bfe3-94b504800901 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:23.834376 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1426489-b5fb-4b57-bfe3-94b504800901 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:23.834655 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1426489-b5fb-4b57-bfe3-94b504800901 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:23.836014 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1426489-b5fb-4b57-bfe3-94b504800901 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:23.847470 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 549/1079] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:04:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:23.853164 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1426489-b5fb-4b57-bfe3-94b504800901 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:23.854270 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb3ffd0-6b8f-47f9-983f-a6bd196e6644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:23.855004 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb3ffd0-6b8f-47f9-983f-a6bd196e6644 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:23.855367 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb3ffd0-6b8f-47f9-983f-a6bd196e6644 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:23.855729 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb3ffd0-6b8f-47f9-983f-a6bd196e6644 None admin] PATH_INFO: `/v3/projects/3c1d8bb4e7ad45e28b1040214a1a5e1a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:23.857019 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb3ffd0-6b8f-47f9-983f-a6bd196e6644 None admin] RBAC: Authorizing `identity:get_project(project_id=3c1d8bb4e7ad45e28b1040214a1a5e1a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:23.858559 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb3ffd0-6b8f-47f9-983f-a6bd196e6644 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:23.860219 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 531/1080] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 6 16:04:23 2022] GET /identity/v3/projects/3c1d8bb4e7ad45e28b1040214a1a5e1a => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:04:23.866129 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb3ffd0-6b8f-47f9-983f-a6bd196e6644 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:23.949195 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1426489-b5fb-4b57-bfe3-94b504800901 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:23.962185 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb3ffd0-6b8f-47f9-983f-a6bd196e6644 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:23.962601 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:23.984590 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0a705c-c589-4ce0-9856-8a339afbb1da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:23.985289 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0a705c-c589-4ce0-9856-8a339afbb1da None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:23.985546 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0a705c-c589-4ce0-9856-8a339afbb1da None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:23.985809 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0a705c-c589-4ce0-9856-8a339afbb1da None admin] PATH_INFO: `/v3/projects/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:23.988686 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:23.992184 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0a705c-c589-4ce0-9856-8a339afbb1da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:23.993671 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0a705c-c589-4ce0-9856-8a339afbb1da None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:23.997085 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cfd323-0b4c-489a-a927-f8b1af3d9f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:23.997735 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cfd323-0b4c-489a-a927-f8b1af3d9f6a None admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:23.998007 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cfd323-0b4c-489a-a927-f8b1af3d9f6a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:23.998276 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cfd323-0b4c-489a-a927-f8b1af3d9f6a None admin] PATH_INFO: `/v3/projects/3c1d8bb4e7ad45e28b1040214a1a5e1a/users/85cf8128739f4389ab4e26559f94877d/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:24.000135 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cfd323-0b4c-489a-a927-f8b1af3d9f6a None admin] RBAC: Authorizing `identity:create_grant(project_id=3c1d8bb4e7ad45e28b1040214a1a5e1a, user_id=85cf8128739f4389ab4e26559f94877d, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:24.001642 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cfd323-0b4c-489a-a927-f8b1af3d9f6a None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:24.005364 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-cd0a705c-c589-4ce0-9856-8a339afbb1da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:04:24.005364 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:24.005364 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:24.005364 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:24.005364 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:24.005364 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:24.005364 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:24.005364 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:24.005364 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:24.005364 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:24.005364 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:24.005364 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:24.005364 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:04:24.005364 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:04:24.005364 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:04:24.006755 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:04:24.006755 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:24.006755 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:24.006755 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:24.006755 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:24.006755 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:24.006755 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:24.006755 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:24.006755 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:24.006755 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:24.006755 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:24.006755 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:24.006755 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:24.006755 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:24.006755 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:24.006755 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:24.006755 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:24.007977 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:04:24.007977 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:04:24.007977 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:04:24.007977 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:04:24.007977 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:04:24.007977 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:04:24.007977 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:04:24.007977 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:04:24.007977 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 550/1081] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 6 16:04:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:04:24.012396 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0a705c-c589-4ce0-9856-8a339afbb1da None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:24.015973 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 532/1082] 10.222.0.43 () {58 vars in 1439 bytes} [Tue Sep 6 16:04:23 2022] PUT /identity/v3/projects/3c1d8bb4e7ad45e28b1040214a1a5e1a/users/85cf8128739f4389ab4e26559f94877d/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:04:24.119968 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0a705c-c589-4ce0-9856-8a339afbb1da None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:24.157568 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4f4ef2-fe5d-49a1-9800-b1de5e7e11c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:24.158455 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4f4ef2-fe5d-49a1-9800-b1de5e7e11c8 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:24.158732 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4f4ef2-fe5d-49a1-9800-b1de5e7e11c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:24.159002 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4f4ef2-fe5d-49a1-9800-b1de5e7e11c8 None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:24.159724 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4f4ef2-fe5d-49a1-9800-b1de5e7e11c8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:24.160009 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4f4ef2-fe5d-49a1-9800-b1de5e7e11c8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:24.161503 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4f4ef2-fe5d-49a1-9800-b1de5e7e11c8 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:24.174374 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 551/1083] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 6 16:04:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:04:24.180593 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cfd323-0b4c-489a-a927-f8b1af3d9f6a None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:24.282929 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cfd323-0b4c-489a-a927-f8b1af3d9f6a None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:24.316326 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f85b1a7-5b4b-4684-b39a-7211141b07e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:24.317017 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f85b1a7-5b4b-4684-b39a-7211141b07e3 None admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:24.317294 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f85b1a7-5b4b-4684-b39a-7211141b07e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:24.317564 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f85b1a7-5b4b-4684-b39a-7211141b07e3 None admin] PATH_INFO: `/v3/projects/128d210e8dad48ac921ed4cb1d444ceb/users/63de41f4348243998029f7d07aa5dc40/roles/012f11d05b68448696f577c7ab2161a8` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:24.319828 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f85b1a7-5b4b-4684-b39a-7211141b07e3 None admin] RBAC: Authorizing `identity:create_grant(project_id=128d210e8dad48ac921ed4cb1d444ceb, user_id=63de41f4348243998029f7d07aa5dc40, role_id=012f11d05b68448696f577c7ab2161a8)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:24.321345 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f85b1a7-5b4b-4684-b39a-7211141b07e3 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:24.336098 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 533/1084] 10.222.0.43 () {58 vars in 1439 bytes} [Tue Sep 6 16:04:24 2022] PUT /identity/v3/projects/128d210e8dad48ac921ed4cb1d444ceb/users/63de41f4348243998029f7d07aa5dc40/roles/012f11d05b68448696f577c7ab2161a8 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:04:25.890281 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107c9598-b366-41d3-8edf-7c48979240c3 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:25.890860 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107c9598-b366-41d3-8edf-7c48979240c3 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:25.890860 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107c9598-b366-41d3-8edf-7c48979240c3 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:25.892298 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 552/1085] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:04:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:04:25.897552 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8c4df3-b661-46b6-a3bd-b1cae6797594 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:25.898019 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8c4df3-b661-46b6-a3bd-b1cae6797594 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:25.898107 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8c4df3-b661-46b6-a3bd-b1cae6797594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:25.922907 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-fd8c4df3-b661-46b6-a3bd-b1cae6797594 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:04:25.924074 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-fd8c4df3-b661-46b6-a3bd-b1cae6797594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:04:25.969483 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:25.994670 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 534/1086] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:04:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:04:26.204984 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99353456-6b63-4d53-8c73-c978a8d7fde3 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:26.205179 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99353456-6b63-4d53-8c73-c978a8d7fde3 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:26.205401 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99353456-6b63-4d53-8c73-c978a8d7fde3 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:26.206803 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 553/1087] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:04:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:04:26.212040 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160f0d6e-37af-4d01-8048-c3db629c9873 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:26.212323 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160f0d6e-37af-4d01-8048-c3db629c9873 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:26.212594 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160f0d6e-37af-4d01-8048-c3db629c9873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:26.234096 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-160f0d6e-37af-4d01-8048-c3db629c9873 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:04:26.235321 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-160f0d6e-37af-4d01-8048-c3db629c9873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:04:26.265342 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99353456-6b63-4d53-8c73-c978a8d7fde3 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:26.296855 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:26.308400 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 535/1088] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:04:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:04:26.369020 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99353456-6b63-4d53-8c73-c978a8d7fde3 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:26.403069 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2592b574-4a01-4e15-81e5-b3e21d0b8be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:26.403693 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2592b574-4a01-4e15-81e5-b3e21d0b8be8 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:26.403968 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2592b574-4a01-4e15-81e5-b3e21d0b8be8 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:26.404235 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2592b574-4a01-4e15-81e5-b3e21d0b8be8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:26.414600 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2592b574-4a01-4e15-81e5-b3e21d0b8be8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:04:26.419414 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-2592b574-4a01-4e15-81e5-b3e21d0b8be8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:04:26.419414 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:26.419414 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:26.419414 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:26.419414 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:26.419414 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:26.419414 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:26.419414 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:26.419414 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:26.419414 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:26.419414 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:26.419414 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:26.419414 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:04:26.419414 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:04:26.419414 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:04:26.419414 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:04:26.420406 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:04:26.420406 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:04:26.420406 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:04:26.420406 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:04:26.420406 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:26.420406 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:26.420406 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:26.420406 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:26.420406 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:26.420406 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:26.420406 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:26.420406 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:26.420406 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:26.420406 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:26.420406 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:26.420406 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:26.421300 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:26.421300 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:26.421300 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:26.421300 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:26.421300 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:04:26.421300 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:04:26.421300 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:04:26.421300 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:04:26.421300 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:04:26.421300 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:04:26.421300 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:04:26.421300 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:04:26.421300 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 554/1089] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 6 16:04:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:04:26.426837 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160f0d6e-37af-4d01-8048-c3db629c9873 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:26.532637 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160f0d6e-37af-4d01-8048-c3db629c9873 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:26.542639 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:26.572172 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ced4ba6-8621-45c4-9830-a1bbe963baae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:26.572797 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ced4ba6-8621-45c4-9830-a1bbe963baae None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:26.573099 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ced4ba6-8621-45c4-9830-a1bbe963baae None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:26.573354 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ced4ba6-8621-45c4-9830-a1bbe963baae None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:26.574030 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ced4ba6-8621-45c4-9830-a1bbe963baae None admin] RBAC: Adding query filter params (name=admin) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:26.574331 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ced4ba6-8621-45c4-9830-a1bbe963baae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:26.575727 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ced4ba6-8621-45c4-9830-a1bbe963baae None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:26.576495 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2592b574-4a01-4e15-81e5-b3e21d0b8be8 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:26.585207 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 536/1090] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 6 16:04:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:26.590886 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ced4ba6-8621-45c4-9830-a1bbe963baae None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:26.669932 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:26.680106 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2592b574-4a01-4e15-81e5-b3e21d0b8be8 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:26.691085 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ced4ba6-8621-45c4-9830-a1bbe963baae None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:26.715994 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176e4d5c-42a1-4358-8991-f55506952e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:26.716628 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176e4d5c-42a1-4358-8991-f55506952e72 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:26.716954 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176e4d5c-42a1-4358-8991-f55506952e72 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:26.717223 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176e4d5c-42a1-4358-8991-f55506952e72 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:26.725487 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaad338-8ecd-49d7-b073-349a0e672f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:26.726116 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaad338-8ecd-49d7-b073-349a0e672f4d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:26.726419 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaad338-8ecd-49d7-b073-349a0e672f4d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:26.726663 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaad338-8ecd-49d7-b073-349a0e672f4d None admin] PATH_INFO: `/v3/users/85cf8128739f4389ab4e26559f94877d` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:26.727204 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176e4d5c-42a1-4358-8991-f55506952e72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:04:26.727791 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaad338-8ecd-49d7-b073-349a0e672f4d None admin] RBAC: Authorizing `identity:get_user(user_id=85cf8128739f4389ab4e26559f94877d)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:26.729271 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaad338-8ecd-49d7-b073-349a0e672f4d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:26.731034 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 537/1091] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 6 16:04:26 2022] GET /identity/v3/users/85cf8128739f4389ab4e26559f94877d => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:04:26.731971 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-176e4d5c-42a1-4358-8991-f55506952e72 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 16:04:26.731971 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:26.731971 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:26.731971 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:26.731971 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:26.731971 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:26.731971 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:26.731971 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:26.731971 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:26.731971 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:26.731971 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:26.731971 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:26.731971 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:04:26.731971 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:04:26.731971 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:04:26.731971 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:04:26.733114 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:04:26.733114 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:04:26.733114 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:04:26.733114 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:04:26.733114 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:26.733114 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:26.733114 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:26.733114 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:26.733114 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:26.733114 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:26.733114 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:26.733114 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:26.733114 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:26.733114 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:26.733114 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:26.733114 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:26.733964 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:26.733964 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:26.733964 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:26.733964 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:26.733964 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:04:26.733964 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:04:26.733964 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:04:26.733964 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:04:26.733964 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:04:26.733964 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:04:26.733964 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 16:04:26.733964 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:04:26.733964 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 555/1092] 10.222.0.43 () {56 vars in 1120 bytes} [Tue Sep 6 16:04:26 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:04:26.736789 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaad338-8ecd-49d7-b073-349a0e672f4d None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:26.738968 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176e4d5c-42a1-4358-8991-f55506952e72 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:26.778306 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:26.838260 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaad338-8ecd-49d7-b073-349a0e672f4d None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:26.838778 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176e4d5c-42a1-4358-8991-f55506952e72 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:26.871521 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc4f9cb-cc48-4f7b-b799-d939eebafd43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:26.871698 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb17a794-55b5-4d24-8bc2-9178f48f209f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:26.872125 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc4f9cb-cc48-4f7b-b799-d939eebafd43 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:26.872253 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb17a794-55b5-4d24-8bc2-9178f48f209f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:26.872399 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc4f9cb-cc48-4f7b-b799-d939eebafd43 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:26.872493 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb17a794-55b5-4d24-8bc2-9178f48f209f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:26.872651 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc4f9cb-cc48-4f7b-b799-d939eebafd43 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:26.872748 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb17a794-55b5-4d24-8bc2-9178f48f209f None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:26.873426 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb17a794-55b5-4d24-8bc2-9178f48f209f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:26.873722 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb17a794-55b5-4d24-8bc2-9178f48f209f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:26.875114 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb17a794-55b5-4d24-8bc2-9178f48f209f None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:26.886005 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 556/1093] 10.222.0.43 () {56 vars in 1115 bytes} [Tue Sep 6 16:04:26 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:04:26.892737 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb17a794-55b5-4d24-8bc2-9178f48f209f None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:26.909418 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:26.946206 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc4f9cb-cc48-4f7b-b799-d939eebafd43 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:26.947682 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc4f9cb-cc48-4f7b-b799-d939eebafd43 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:26.985138 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:26.997296 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb17a794-55b5-4d24-8bc2-9178f48f209f None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:27.031020 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6b5895-88b2-4899-af11-d66d24343973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:27.031648 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6b5895-88b2-4899-af11-d66d24343973 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:27.031912 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6b5895-88b2-4899-af11-d66d24343973 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:27.032246 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6b5895-88b2-4899-af11-d66d24343973 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:27.042439 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6b5895-88b2-4899-af11-d66d24343973 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:27.043870 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6b5895-88b2-4899-af11-d66d24343973 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:27.044424 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 538/1094] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 6 16:04:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 16:04:27.050453 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc4f9cb-cc48-4f7b-b799-d939eebafd43 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:27.057752 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-bd6b5895-88b2-4899-af11-d66d24343973 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:04:27.057752 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:27.057752 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:27.057752 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:27.057752 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:27.057752 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:27.057752 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:27.057752 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:27.057752 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:27.057752 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:27.057752 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:27.057752 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:27.057752 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:04:27.057752 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:04:27.057752 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:04:27.057752 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:04:27.058704 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:27.058704 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:27.058704 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:27.058704 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:27.058704 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:27.058704 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:27.058704 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:27.058704 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:27.058704 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:27.058704 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:27.058704 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:27.058704 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:27.058704 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:27.058704 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:27.058704 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:27.058704 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:27.058704 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:04:27.064791 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:04:27.064791 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:04:27.064791 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:04:27.064791 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 557/1095] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:04:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:04:27.064791 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6b5895-88b2-4899-af11-d66d24343973 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:27.128277 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:27.150942 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc4f9cb-cc48-4f7b-b799-d939eebafd43 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:27.164076 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6b5895-88b2-4899-af11-d66d24343973 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:27.183987 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2973c91e-2571-4d5a-bc56-db1e51dc8113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:27.184618 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973c91e-2571-4d5a-bc56-db1e51dc8113 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:27.184923 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973c91e-2571-4d5a-bc56-db1e51dc8113 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:27.185193 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973c91e-2571-4d5a-bc56-db1e51dc8113 None admin] PATH_INFO: `/v3/projects/3c1d8bb4e7ad45e28b1040214a1a5e1a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:27.186292 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2973c91e-2571-4d5a-bc56-db1e51dc8113 None admin] RBAC: Authorizing `identity:get_project(project_id=3c1d8bb4e7ad45e28b1040214a1a5e1a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:27.187762 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2973c91e-2571-4d5a-bc56-db1e51dc8113 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:27.189463 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 539/1096] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 6 16:04:27 2022] GET /identity/v3/projects/3c1d8bb4e7ad45e28b1040214a1a5e1a => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:04:27.195516 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2973c91e-2571-4d5a-bc56-db1e51dc8113 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:27.197337 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7d3736-73b4-441b-a058-b3e2d4505961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:27.197939 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7d3736-73b4-441b-a058-b3e2d4505961 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:27.198210 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7d3736-73b4-441b-a058-b3e2d4505961 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:27.198476 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7d3736-73b4-441b-a058-b3e2d4505961 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:27.199098 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7d3736-73b4-441b-a058-b3e2d4505961 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:27.199393 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7d3736-73b4-441b-a058-b3e2d4505961 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:27.200796 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7d3736-73b4-441b-a058-b3e2d4505961 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:27.204156 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:27.215093 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 558/1097] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:04:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:27.221279 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7d3736-73b4-441b-a058-b3e2d4505961 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:27.297923 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2973c91e-2571-4d5a-bc56-db1e51dc8113 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:27.318453 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7d3736-73b4-441b-a058-b3e2d4505961 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:27.331507 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56681569-afb2-44d2-b9ef-2a317c49664d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:27.332121 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56681569-afb2-44d2-b9ef-2a317c49664d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:27.332409 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56681569-afb2-44d2-b9ef-2a317c49664d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:27.332681 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56681569-afb2-44d2-b9ef-2a317c49664d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:27.333455 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56681569-afb2-44d2-b9ef-2a317c49664d None admin] RBAC: Adding query filter params (role.id=edaabdea8be84a7980fb2c651fe52f63, scope.project.id=3c1d8bb4e7ad45e28b1040214a1a5e1a, user.id=85cf8128739f4389ab4e26559f94877d) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:27.333764 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56681569-afb2-44d2-b9ef-2a317c49664d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:27.335214 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56681569-afb2-44d2-b9ef-2a317c49664d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:27.340879 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 540/1098] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 6 16:04:27 2022] GET /identity/v3/role_assignments?user.id=85cf8128739f4389ab4e26559f94877d&scope.project.id=3c1d8bb4e7ad45e28b1040214a1a5e1a&role.id=edaabdea8be84a7980fb2c651fe52f63 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:04:27.346882 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:27.356499 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84c1464-55d7-469d-b738-51bdbabe5111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:27.357194 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84c1464-55d7-469d-b738-51bdbabe5111 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:27.357464 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84c1464-55d7-469d-b738-51bdbabe5111 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:27.357728 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84c1464-55d7-469d-b738-51bdbabe5111 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:27.366120 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84c1464-55d7-469d-b738-51bdbabe5111 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:27.367621 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84c1464-55d7-469d-b738-51bdbabe5111 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:27.380840 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-e84c1464-55d7-469d-b738-51bdbabe5111 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 16:04:27.380840 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:27.380840 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:27.380840 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:27.380840 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:27.380840 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:27.380840 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:27.380840 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:27.380840 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:27.380840 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:27.380840 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:27.380840 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:27.380840 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:04:27.380840 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:04:27.380840 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:04:27.380840 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:04:27.381865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:27.381865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:27.381865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:04:27.381865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:04:27.381865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:04:27.381865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:04:27.381865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:27.381865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:27.381865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:27.381865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:27.381865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:27.381865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:27.381865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:27.381865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:27.381865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:27.381865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:27.381865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:27.381865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:27.382841 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:27.382841 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:27.382841 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:04:27.382841 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:04:27.382841 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:04:27.382841 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:04:27.382841 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:04:27.382841 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:04:27.382841 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 16:04:27.382841 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:04:27.383652 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 559/1099] 10.222.0.43 () {56 vars in 1093 bytes} [Tue Sep 6 16:04:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:04:27.388104 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56681569-afb2-44d2-b9ef-2a317c49664d None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:27.463560 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:27.503141 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56681569-afb2-44d2-b9ef-2a317c49664d None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:27.536780 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34b6b7f-5bc9-40e8-9c49-87667dbcec05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:27.537386 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34b6b7f-5bc9-40e8-9c49-87667dbcec05 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:27.537659 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34b6b7f-5bc9-40e8-9c49-87667dbcec05 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:27.537924 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34b6b7f-5bc9-40e8-9c49-87667dbcec05 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:27.538652 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34b6b7f-5bc9-40e8-9c49-87667dbcec05 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:27.538948 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34b6b7f-5bc9-40e8-9c49-87667dbcec05 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:27.540398 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34b6b7f-5bc9-40e8-9c49-87667dbcec05 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:27.557471 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 541/1100] 10.222.0.43 () {56 vars in 1133 bytes} [Tue Sep 6 16:04:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:27.563365 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84c1464-55d7-469d-b738-51bdbabe5111 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:27.663558 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84c1464-55d7-469d-b738-51bdbabe5111 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:27.697359 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b2c342-836e-4501-9a3a-b0e6383d5d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:27.697976 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b2c342-836e-4501-9a3a-b0e6383d5d81 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:27.698246 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b2c342-836e-4501-9a3a-b0e6383d5d81 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:27.698495 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b2c342-836e-4501-9a3a-b0e6383d5d81 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:27.752315 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:27.768229 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b2c342-836e-4501-9a3a-b0e6383d5d81 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:27.769748 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b2c342-836e-4501-9a3a-b0e6383d5d81 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:27.869847 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 560/1101] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 6 16:04:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 16:04:27.876166 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34b6b7f-5bc9-40e8-9c49-87667dbcec05 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:27.978076 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34b6b7f-5bc9-40e8-9c49-87667dbcec05 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:28.004895 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:28.013471 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c9250c-c642-41eb-b767-c0c168df1d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:28.014102 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c9250c-c642-41eb-b767-c0c168df1d4a None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:28.014381 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c9250c-c642-41eb-b767-c0c168df1d4a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:28.014651 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c9250c-c642-41eb-b767-c0c168df1d4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:28.024711 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c9250c-c642-41eb-b767-c0c168df1d4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:28.026208 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c9250c-c642-41eb-b767-c0c168df1d4a None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:28.040891 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-d5c9250c-c642-41eb-b767-c0c168df1d4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:04:28.040891 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:28.040891 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:28.040891 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:28.040891 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:28.040891 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:28.040891 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:28.040891 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:28.040891 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:28.040891 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:28.040891 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:28.040891 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:28.040891 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:04:28.040891 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:04:28.040891 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:04:28.040891 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:04:28.042015 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:28.042015 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:28.042015 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:28.042015 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:28.042015 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:28.042015 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:28.042015 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:28.042015 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:28.042015 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:28.042015 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:28.042015 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:28.042015 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:28.042015 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:28.042015 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:28.042015 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:28.042015 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:28.042015 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:04:28.043079 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:04:28.043079 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:04:28.043079 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:04:28.043079 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 542/1102] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:04:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:04:28.048089 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b2c342-836e-4501-9a3a-b0e6383d5d81 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:28.135947 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:28.151989 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b2c342-836e-4501-9a3a-b0e6383d5d81 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:28.186671 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf90a41-b029-467a-8cad-91eda2270393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:28.187266 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf90a41-b029-467a-8cad-91eda2270393 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:28.187542 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf90a41-b029-467a-8cad-91eda2270393 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:28.187856 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf90a41-b029-467a-8cad-91eda2270393 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:28.188556 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf90a41-b029-467a-8cad-91eda2270393 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:28.188890 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf90a41-b029-467a-8cad-91eda2270393 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:28.190268 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf90a41-b029-467a-8cad-91eda2270393 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:28.202280 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 561/1103] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:04:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:28.208420 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c9250c-c642-41eb-b767-c0c168df1d4a None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:28.317509 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c9250c-c642-41eb-b767-c0c168df1d4a None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:28.352867 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1968ad1-b9d1-4524-b832-b127286453df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:28.353504 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1968ad1-b9d1-4524-b832-b127286453df None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:28.353780 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1968ad1-b9d1-4524-b832-b127286453df None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:28.354049 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1968ad1-b9d1-4524-b832-b127286453df None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:28.360701 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1968ad1-b9d1-4524-b832-b127286453df None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:28.362233 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1968ad1-b9d1-4524-b832-b127286453df None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:28.374486 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-b1968ad1-b9d1-4524-b832-b127286453df None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:04:28.374486 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:28.374486 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:28.374486 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:28.374486 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:28.374486 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:28.374486 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:28.374486 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:28.374486 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:28.374486 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:28.374486 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:28.374486 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:28.374486 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:04:28.374486 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:04:28.374486 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:04:28.375403 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:04:28.375403 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:28.375403 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:28.375403 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:28.375403 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:28.375403 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:28.375403 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:28.375403 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:28.375403 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:28.375403 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:28.375403 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:28.375403 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:28.375403 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:28.375403 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:28.375403 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:28.375403 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:28.375403 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:28.376399 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:04:28.376399 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:04:28.376399 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:04:28.376399 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:04:28.376399 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:04:28.376399 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:04:28.376399 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:04:28.376399 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:04:28.376399 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 543/1104] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 6 16:04:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:04:28.381798 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf90a41-b029-467a-8cad-91eda2270393 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:28.487627 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf90a41-b029-467a-8cad-91eda2270393 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:28.521270 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d222ed25-b8c1-4aeb-9990-25460230eb22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:28.521869 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d222ed25-b8c1-4aeb-9990-25460230eb22 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:28.522147 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d222ed25-b8c1-4aeb-9990-25460230eb22 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:28.522418 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d222ed25-b8c1-4aeb-9990-25460230eb22 None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:28.523099 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d222ed25-b8c1-4aeb-9990-25460230eb22 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:28.523399 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d222ed25-b8c1-4aeb-9990-25460230eb22 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:28.524865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d222ed25-b8c1-4aeb-9990-25460230eb22 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:28.536699 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 562/1105] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 6 16:04:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:28.542888 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1968ad1-b9d1-4524-b832-b127286453df None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:28.585460 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:28.646310 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1968ad1-b9d1-4524-b832-b127286453df None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:28.680303 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f583aea-bae3-4828-aafb-84225b2d095d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:28.680956 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f583aea-bae3-4828-aafb-84225b2d095d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:28.681227 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f583aea-bae3-4828-aafb-84225b2d095d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:28.681485 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f583aea-bae3-4828-aafb-84225b2d095d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:28.682212 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f583aea-bae3-4828-aafb-84225b2d095d None admin] RBAC: Adding query filter params (role.id=012f11d05b68448696f577c7ab2161a8, scope.project.id=128d210e8dad48ac921ed4cb1d444ceb, user.id=63de41f4348243998029f7d07aa5dc40) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:28.682509 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f583aea-bae3-4828-aafb-84225b2d095d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:28.684050 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f583aea-bae3-4828-aafb-84225b2d095d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:28.689917 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 544/1106] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 6 16:04:28 2022] GET /identity/v3/role_assignments?user.id=63de41f4348243998029f7d07aa5dc40&scope.project.id=128d210e8dad48ac921ed4cb1d444ceb&role.id=012f11d05b68448696f577c7ab2161a8 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:29.253009 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13832fc-1a06-473a-8403-33634fdb541a None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:29.253559 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13832fc-1a06-473a-8403-33634fdb541a None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:29.253559 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13832fc-1a06-473a-8403-33634fdb541a None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:29.254915 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 563/1107] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:04:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:04:29.260421 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42329ce5-97a9-4eea-b383-b46e8921c0d7 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:29.260868 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42329ce5-97a9-4eea-b383-b46e8921c0d7 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:29.261008 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42329ce5-97a9-4eea-b383-b46e8921c0d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:29.280626 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-42329ce5-97a9-4eea-b383-b46e8921c0d7 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:04:29.282028 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-42329ce5-97a9-4eea-b383-b46e8921c0d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:04:29.353877 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 545/1108] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:04:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:04:29.616905 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13832fc-1a06-473a-8403-33634fdb541a None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:29.632211 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:29.737510 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13832fc-1a06-473a-8403-33634fdb541a None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:29.773891 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3f18d7-b6ab-4dee-9883-d69a67219a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:29.774506 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3f18d7-b6ab-4dee-9883-d69a67219a9a None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:29.774784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3f18d7-b6ab-4dee-9883-d69a67219a9a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:29.775054 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3f18d7-b6ab-4dee-9883-d69a67219a9a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:29.776203 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3f18d7-b6ab-4dee-9883-d69a67219a9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:29.777677 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3f18d7-b6ab-4dee-9883-d69a67219a9a None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:29.779236 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 564/1109] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:04:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:29.785440 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42329ce5-97a9-4eea-b383-b46e8921c0d7 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:29.828343 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:29.894925 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42329ce5-97a9-4eea-b383-b46e8921c0d7 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:29.930154 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057ea4e8-41cb-4132-8d14-712dd580b2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:29.930748 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057ea4e8-41cb-4132-8d14-712dd580b2fb None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:29.931025 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057ea4e8-41cb-4132-8d14-712dd580b2fb None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:29.931292 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057ea4e8-41cb-4132-8d14-712dd580b2fb None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:29.931962 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057ea4e8-41cb-4132-8d14-712dd580b2fb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:29.933417 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057ea4e8-41cb-4132-8d14-712dd580b2fb None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:29.957848 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 546/1110] 10.222.0.43 () {60 vars in 1132 bytes} [Tue Sep 6 16:04:29 2022] POST /identity/v3/users => generated 286 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:04:31.864651 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743c1e45-a102-4f7d-bc68-9655293804c0 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:31.865545 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743c1e45-a102-4f7d-bc68-9655293804c0 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:31.865545 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743c1e45-a102-4f7d-bc68-9655293804c0 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:31.867792 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 565/1111] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:04:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:04:31.876806 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e7559e-1cda-436b-a4f4-15ccd8091149 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:31.877194 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e7559e-1cda-436b-a4f4-15ccd8091149 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:31.877401 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e7559e-1cda-436b-a4f4-15ccd8091149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:31.901541 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-26e7559e-1cda-436b-a4f4-15ccd8091149 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:04:31.902733 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-26e7559e-1cda-436b-a4f4-15ccd8091149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:04:31.976696 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 547/1112] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:04:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:04:32.252291 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743c1e45-a102-4f7d-bc68-9655293804c0 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:32.334655 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:32.362030 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743c1e45-a102-4f7d-bc68-9655293804c0 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:32.397494 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e9d65b-4281-4a6a-8625-4a66b7fe8f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:32.398111 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9d65b-4281-4a6a-8625-4a66b7fe8f77 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:32.398385 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9d65b-4281-4a6a-8625-4a66b7fe8f77 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:32.398651 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9d65b-4281-4a6a-8625-4a66b7fe8f77 None admin] PATH_INFO: `/v3/roles/61df19b017494bdcadc9a5e9d8bb9e52` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:32.400141 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e9d65b-4281-4a6a-8625-4a66b7fe8f77 None admin] RBAC: Authorizing `identity:get_role(role_id=61df19b017494bdcadc9a5e9d8bb9e52)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:32.401554 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e9d65b-4281-4a6a-8625-4a66b7fe8f77 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:32.402865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 566/1113] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 6 16:04:32 2022] GET /identity/v3/roles/61df19b017494bdcadc9a5e9d8bb9e52 => generated 223 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:32.409932 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e7559e-1cda-436b-a4f4-15ccd8091149 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:32.436816 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:32.519636 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e7559e-1cda-436b-a4f4-15ccd8091149 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:32.554702 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4683fc-77d7-4c9e-89df-122264fd8a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:32.555313 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4683fc-77d7-4c9e-89df-122264fd8a3c None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:32.555592 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4683fc-77d7-4c9e-89df-122264fd8a3c None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:32.555860 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4683fc-77d7-4c9e-89df-122264fd8a3c None admin] PATH_INFO: `/v3/users/78f35882a8be47be9066e1d2c1b0639e` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:32.569420 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4683fc-77d7-4c9e-89df-122264fd8a3c None admin] RBAC: Authorizing `identity:get_user(user_id=78f35882a8be47be9066e1d2c1b0639e)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:32.570873 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4683fc-77d7-4c9e-89df-122264fd8a3c None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:32.572460 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 548/1114] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 6 16:04:32 2022] GET /identity/v3/users/78f35882a8be47be9066e1d2c1b0639e => generated 286 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:32.579224 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e9d65b-4281-4a6a-8625-4a66b7fe8f77 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:32.694808 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e9d65b-4281-4a6a-8625-4a66b7fe8f77 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:32.733899 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb62f56-4dc8-4735-a406-bec4ac224e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:32.734829 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb62f56-4dc8-4735-a406-bec4ac224e00 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:32.735230 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb62f56-4dc8-4735-a406-bec4ac224e00 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:32.735672 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb62f56-4dc8-4735-a406-bec4ac224e00 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:32.747704 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:32.803531 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb62f56-4dc8-4735-a406-bec4ac224e00 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:32.804956 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb62f56-4dc8-4735-a406-bec4ac224e00 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:32.907221 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 567/1115] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 6 16:04:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 16:04:32.916275 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4683fc-77d7-4c9e-89df-122264fd8a3c None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:32.998379 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:33.020937 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4683fc-77d7-4c9e-89df-122264fd8a3c None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:33.055769 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0551a38-cb87-4c0a-9277-399ae174b0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:33.056322 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0551a38-cb87-4c0a-9277-399ae174b0a9 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:33.056594 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0551a38-cb87-4c0a-9277-399ae174b0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:33.056890 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0551a38-cb87-4c0a-9277-399ae174b0a9 None admin] PATH_INFO: `/v3/projects/3c1d8bb4e7ad45e28b1040214a1a5e1a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:33.058043 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0551a38-cb87-4c0a-9277-399ae174b0a9 None admin] RBAC: Authorizing `identity:get_project(project_id=3c1d8bb4e7ad45e28b1040214a1a5e1a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:33.059485 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0551a38-cb87-4c0a-9277-399ae174b0a9 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:33.061136 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 549/1116] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 6 16:04:32 2022] GET /identity/v3/projects/3c1d8bb4e7ad45e28b1040214a1a5e1a => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:04:33.068261 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb62f56-4dc8-4735-a406-bec4ac224e00 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:33.115061 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:33.183352 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb62f56-4dc8-4735-a406-bec4ac224e00 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:33.220963 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5407e81-37b9-4af5-8aec-2e5021f8adb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:33.221548 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5407e81-37b9-4af5-8aec-2e5021f8adb3 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:33.221823 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5407e81-37b9-4af5-8aec-2e5021f8adb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:33.222091 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5407e81-37b9-4af5-8aec-2e5021f8adb3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:33.222803 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5407e81-37b9-4af5-8aec-2e5021f8adb3 None admin] RBAC: Adding query filter params (role.id=61df19b017494bdcadc9a5e9d8bb9e52, scope.project.id=3c1d8bb4e7ad45e28b1040214a1a5e1a, user.id=78f35882a8be47be9066e1d2c1b0639e) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:33.223112 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5407e81-37b9-4af5-8aec-2e5021f8adb3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:33.224534 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5407e81-37b9-4af5-8aec-2e5021f8adb3 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:33.230469 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 568/1117] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 6 16:04:33 2022] GET /identity/v3/role_assignments?user.id=78f35882a8be47be9066e1d2c1b0639e&scope.project.id=3c1d8bb4e7ad45e28b1040214a1a5e1a&role.id=61df19b017494bdcadc9a5e9d8bb9e52 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:35.084174 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d29bdc7-9ed6-4890-be66-009f806671f9 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:35.085099 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d29bdc7-9ed6-4890-be66-009f806671f9 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:35.085099 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d29bdc7-9ed6-4890-be66-009f806671f9 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:35.087259 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 550/1118] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:04:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:04:35.092728 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b86d726-0ec7-489f-97c9-4bce8ac16437 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:35.093210 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b86d726-0ec7-489f-97c9-4bce8ac16437 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:35.093304 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b86d726-0ec7-489f-97c9-4bce8ac16437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:35.113254 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-9b86d726-0ec7-489f-97c9-4bce8ac16437 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:04:35.114440 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-9b86d726-0ec7-489f-97c9-4bce8ac16437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:04:35.183411 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 569/1119] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:04:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:04:35.450569 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d29bdc7-9ed6-4890-be66-009f806671f9 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:35.568376 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d29bdc7-9ed6-4890-be66-009f806671f9 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:35.602224 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbb70af-c724-41aa-b997-56d57ef8ab69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:35.602869 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb70af-c724-41aa-b997-56d57ef8ab69 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:35.603139 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb70af-c724-41aa-b997-56d57ef8ab69 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:35.603402 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb70af-c724-41aa-b997-56d57ef8ab69 None admin] PATH_INFO: `/v3/roles/61df19b017494bdcadc9a5e9d8bb9e52` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:35.604925 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbb70af-c724-41aa-b997-56d57ef8ab69 None admin] RBAC: Authorizing `identity:get_role(role_id=61df19b017494bdcadc9a5e9d8bb9e52)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:35.606305 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbb70af-c724-41aa-b997-56d57ef8ab69 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:35.607614 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 551/1120] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 6 16:04:35 2022] GET /identity/v3/roles/61df19b017494bdcadc9a5e9d8bb9e52 => generated 223 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:35.616426 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b86d726-0ec7-489f-97c9-4bce8ac16437 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:35.652655 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:35.724234 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b86d726-0ec7-489f-97c9-4bce8ac16437 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:35.760867 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594e97ce-9583-411c-8a2c-6e73a3f87595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:35.761498 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594e97ce-9583-411c-8a2c-6e73a3f87595 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:35.761775 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594e97ce-9583-411c-8a2c-6e73a3f87595 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:35.762041 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594e97ce-9583-411c-8a2c-6e73a3f87595 None admin] PATH_INFO: `/v3/users/78f35882a8be47be9066e1d2c1b0639e` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:35.763227 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594e97ce-9583-411c-8a2c-6e73a3f87595 None admin] RBAC: Authorizing `identity:get_user(user_id=78f35882a8be47be9066e1d2c1b0639e)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:35.764688 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594e97ce-9583-411c-8a2c-6e73a3f87595 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:35.766318 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 570/1121] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 6 16:04:35 2022] GET /identity/v3/users/78f35882a8be47be9066e1d2c1b0639e => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:35.774981 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbb70af-c724-41aa-b997-56d57ef8ab69 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:35.791403 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:35.885193 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbb70af-c724-41aa-b997-56d57ef8ab69 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:35.925076 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c1b9d1-f4ec-4696-9bae-e7a2c60c5f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:35.925693 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c1b9d1-f4ec-4696-9bae-e7a2c60c5f3d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:35.925966 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c1b9d1-f4ec-4696-9bae-e7a2c60c5f3d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:35.926232 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c1b9d1-f4ec-4696-9bae-e7a2c60c5f3d None admin] PATH_INFO: `/v3/projects/3c1d8bb4e7ad45e28b1040214a1a5e1a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:35.927464 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c1b9d1-f4ec-4696-9bae-e7a2c60c5f3d None admin] RBAC: Authorizing `identity:get_project(project_id=3c1d8bb4e7ad45e28b1040214a1a5e1a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:35.928952 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c1b9d1-f4ec-4696-9bae-e7a2c60c5f3d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:35.930583 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 552/1122] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 6 16:04:35 2022] GET /identity/v3/projects/3c1d8bb4e7ad45e28b1040214a1a5e1a => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:35.937085 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594e97ce-9583-411c-8a2c-6e73a3f87595 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:36.044184 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594e97ce-9583-411c-8a2c-6e73a3f87595 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:36.046579 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:36.089234 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d4869c-9bbc-4fd1-bf85-ecb8dac940b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:36.089911 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d4869c-9bbc-4fd1-bf85-ecb8dac940b9 None admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:36.090190 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d4869c-9bbc-4fd1-bf85-ecb8dac940b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:36.090451 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d4869c-9bbc-4fd1-bf85-ecb8dac940b9 None admin] PATH_INFO: `/v3/projects/3c1d8bb4e7ad45e28b1040214a1a5e1a/users/78f35882a8be47be9066e1d2c1b0639e/roles/61df19b017494bdcadc9a5e9d8bb9e52` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:36.093468 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d4869c-9bbc-4fd1-bf85-ecb8dac940b9 None admin] RBAC: Authorizing `identity:create_grant(project_id=3c1d8bb4e7ad45e28b1040214a1a5e1a, user_id=78f35882a8be47be9066e1d2c1b0639e, role_id=61df19b017494bdcadc9a5e9d8bb9e52)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:36.095614 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d4869c-9bbc-4fd1-bf85-ecb8dac940b9 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:36.114035 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 571/1123] 10.222.0.43 () {58 vars in 1439 bytes} [Tue Sep 6 16:04:35 2022] PUT /identity/v3/projects/3c1d8bb4e7ad45e28b1040214a1a5e1a/users/78f35882a8be47be9066e1d2c1b0639e/roles/61df19b017494bdcadc9a5e9d8bb9e52 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:04:37.987739 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb12d9-d596-4435-af23-2de4aa1713ce None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:37.988687 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb12d9-d596-4435-af23-2de4aa1713ce None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:37.988687 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb12d9-d596-4435-af23-2de4aa1713ce None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:37.990827 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 553/1124] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:04:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:04:37.996406 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69d23ee-39e4-47d0-84b2-fbff71787178 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:37.996886 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69d23ee-39e4-47d0-84b2-fbff71787178 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:37.996984 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69d23ee-39e4-47d0-84b2-fbff71787178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:38.016785 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-c69d23ee-39e4-47d0-84b2-fbff71787178 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:04:38.018006 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c69d23ee-39e4-47d0-84b2-fbff71787178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:04:38.087381 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 572/1125] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:04:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:04:38.352816 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bb12d9-d596-4435-af23-2de4aa1713ce None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:38.440355 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:38.459894 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bb12d9-d596-4435-af23-2de4aa1713ce None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:38.494515 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439e9c0b-a9bb-4bd4-8106-30d81f79770a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:38.495123 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439e9c0b-a9bb-4bd4-8106-30d81f79770a None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:38.495398 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439e9c0b-a9bb-4bd4-8106-30d81f79770a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:38.495661 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439e9c0b-a9bb-4bd4-8106-30d81f79770a None admin] PATH_INFO: `/v3/roles/61df19b017494bdcadc9a5e9d8bb9e52` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:38.497169 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439e9c0b-a9bb-4bd4-8106-30d81f79770a None admin] RBAC: Authorizing `identity:get_role(role_id=61df19b017494bdcadc9a5e9d8bb9e52)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:38.498532 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439e9c0b-a9bb-4bd4-8106-30d81f79770a None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:38.499834 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 554/1126] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 6 16:04:38 2022] GET /identity/v3/roles/61df19b017494bdcadc9a5e9d8bb9e52 => generated 223 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:38.509570 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69d23ee-39e4-47d0-84b2-fbff71787178 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:38.608119 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:38.616628 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69d23ee-39e4-47d0-84b2-fbff71787178 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:38.655695 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc144a7-e91c-4dcf-8475-b3e2d5114e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:38.656315 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc144a7-e91c-4dcf-8475-b3e2d5114e4a None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:38.656587 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc144a7-e91c-4dcf-8475-b3e2d5114e4a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:38.656881 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc144a7-e91c-4dcf-8475-b3e2d5114e4a None admin] PATH_INFO: `/v3/users/78f35882a8be47be9066e1d2c1b0639e` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:38.658045 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc144a7-e91c-4dcf-8475-b3e2d5114e4a None admin] RBAC: Authorizing `identity:get_user(user_id=78f35882a8be47be9066e1d2c1b0639e)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:38.659473 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc144a7-e91c-4dcf-8475-b3e2d5114e4a None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:38.661088 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 573/1127] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 6 16:04:38 2022] GET /identity/v3/users/78f35882a8be47be9066e1d2c1b0639e => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:38.667053 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439e9c0b-a9bb-4bd4-8106-30d81f79770a None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:38.772229 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439e9c0b-a9bb-4bd4-8106-30d81f79770a None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:38.807567 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d7cd97-f509-4c46-b622-0d784c884d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:38.808154 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d7cd97-f509-4c46-b622-0d784c884d3d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:38.808423 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d7cd97-f509-4c46-b622-0d784c884d3d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:38.808688 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d7cd97-f509-4c46-b622-0d784c884d3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:38.827283 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:38.882529 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d7cd97-f509-4c46-b622-0d784c884d3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:38.883920 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d7cd97-f509-4c46-b622-0d784c884d3d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:38.988792 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 555/1128] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 6 16:04:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 16:04:38.992614 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc144a7-e91c-4dcf-8475-b3e2d5114e4a None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:39.093838 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc144a7-e91c-4dcf-8475-b3e2d5114e4a None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:39.141149 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36504546-d4c9-4be8-9be5-67485f318190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:39.141717 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36504546-d4c9-4be8-9be5-67485f318190 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:39.141991 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36504546-d4c9-4be8-9be5-67485f318190 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:39.142259 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36504546-d4c9-4be8-9be5-67485f318190 None admin] PATH_INFO: `/v3/projects/3c1d8bb4e7ad45e28b1040214a1a5e1a` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:39.143461 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36504546-d4c9-4be8-9be5-67485f318190 None admin] RBAC: Authorizing `identity:get_project(project_id=3c1d8bb4e7ad45e28b1040214a1a5e1a)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:39.144937 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36504546-d4c9-4be8-9be5-67485f318190 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:39.146621 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 574/1129] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 6 16:04:38 2022] GET /identity/v3/projects/3c1d8bb4e7ad45e28b1040214a1a5e1a => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:39.153157 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d7cd97-f509-4c46-b622-0d784c884d3d None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:39.207436 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:39.266328 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d7cd97-f509-4c46-b622-0d784c884d3d None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:39.302693 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e58a5b-9f94-4d15-8296-ff04243181ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:39.303301 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e58a5b-9f94-4d15-8296-ff04243181ff None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:39.303582 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e58a5b-9f94-4d15-8296-ff04243181ff None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:39.303846 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e58a5b-9f94-4d15-8296-ff04243181ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:39.304575 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e58a5b-9f94-4d15-8296-ff04243181ff None admin] RBAC: Adding query filter params (role.id=61df19b017494bdcadc9a5e9d8bb9e52, scope.project.id=3c1d8bb4e7ad45e28b1040214a1a5e1a, user.id=78f35882a8be47be9066e1d2c1b0639e) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:39.304909 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e58a5b-9f94-4d15-8296-ff04243181ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:39.306354 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e58a5b-9f94-4d15-8296-ff04243181ff None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:39.312361 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 556/1130] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 6 16:04:39 2022] GET /identity/v3/role_assignments?user.id=78f35882a8be47be9066e1d2c1b0639e&scope.project.id=3c1d8bb4e7ad45e28b1040214a1a5e1a&role.id=61df19b017494bdcadc9a5e9d8bb9e52 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:41.227580 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dfdbfd-de3e-4393-b43c-75248b9997f5 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:41.228205 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dfdbfd-de3e-4393-b43c-75248b9997f5 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:41.228205 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dfdbfd-de3e-4393-b43c-75248b9997f5 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:41.229504 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 575/1131] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:04:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:04:41.235793 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef294e-164c-49ab-8465-6b59b74c468f None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:41.236548 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef294e-164c-49ab-8465-6b59b74c468f None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:41.236645 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef294e-164c-49ab-8465-6b59b74c468f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:41.260706 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-ebef294e-164c-49ab-8465-6b59b74c468f None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:04:41.261915 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ebef294e-164c-49ab-8465-6b59b74c468f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:04:41.337818 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 557/1132] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:04:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:04:41.601863 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dfdbfd-de3e-4393-b43c-75248b9997f5 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:41.640971 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:41.717041 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dfdbfd-de3e-4393-b43c-75248b9997f5 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:41.751375 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c513af-a8f9-4b86-b01e-2ac94a34b9ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:41.751988 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c513af-a8f9-4b86-b01e-2ac94a34b9ca None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:41.752259 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c513af-a8f9-4b86-b01e-2ac94a34b9ca None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:41.752532 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c513af-a8f9-4b86-b01e-2ac94a34b9ca None admin] PATH_INFO: `/v3/domains/default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:41.753678 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c513af-a8f9-4b86-b01e-2ac94a34b9ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:41.755108 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c513af-a8f9-4b86-b01e-2ac94a34b9ca None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:41.756646 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 576/1133] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:04:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:41.762752 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebef294e-164c-49ab-8465-6b59b74c468f None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:41.811612 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:41.875280 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebef294e-164c-49ab-8465-6b59b74c468f None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:41.914091 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c46b12-3976-42bb-9a73-8f9da55465e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:41.914687 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c46b12-3976-42bb-9a73-8f9da55465e4 None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:41.914963 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c46b12-3976-42bb-9a73-8f9da55465e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:41.915232 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c46b12-3976-42bb-9a73-8f9da55465e4 None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:41.915874 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c46b12-3976-42bb-9a73-8f9da55465e4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:41.917366 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c46b12-3976-42bb-9a73-8f9da55465e4 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:41.947163 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 558/1134] 10.222.0.43 () {60 vars in 1141 bytes} [Tue Sep 6 16:04:41 2022] POST /identity/v3/projects => generated 311 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:04:43.883409 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376d10f2-75f0-4d09-9d77-973305faa1d3 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:43.884005 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376d10f2-75f0-4d09-9d77-973305faa1d3 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:43.884005 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376d10f2-75f0-4d09-9d77-973305faa1d3 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:43.885325 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 577/1135] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:04:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:04:43.891748 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e435186-7ed2-4e24-9b45-84b1127e69f1 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:43.892539 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e435186-7ed2-4e24-9b45-84b1127e69f1 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:43.892662 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e435186-7ed2-4e24-9b45-84b1127e69f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:43.922478 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-9e435186-7ed2-4e24-9b45-84b1127e69f1 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:04:43.923669 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-9e435186-7ed2-4e24-9b45-84b1127e69f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:04:43.991774 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 559/1136] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:04:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:04:44.253665 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376d10f2-75f0-4d09-9d77-973305faa1d3 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:44.357096 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:44.362215 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376d10f2-75f0-4d09-9d77-973305faa1d3 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:44.398901 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c7ed63-e532-4cc6-bce3-eb072f90997c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:44.399510 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c7ed63-e532-4cc6-bce3-eb072f90997c None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:44.399782 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c7ed63-e532-4cc6-bce3-eb072f90997c None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:44.400033 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c7ed63-e532-4cc6-bce3-eb072f90997c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:44.401166 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c7ed63-e532-4cc6-bce3-eb072f90997c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:44.402594 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c7ed63-e532-4cc6-bce3-eb072f90997c None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:44.404168 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 578/1137] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:04:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:44.410614 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e435186-7ed2-4e24-9b45-84b1127e69f1 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:44.441629 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:44.517606 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e435186-7ed2-4e24-9b45-84b1127e69f1 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:44.552118 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e7adbe-e59a-4122-925e-ffa0f131701e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:44.552710 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7adbe-e59a-4122-925e-ffa0f131701e None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:44.553014 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7adbe-e59a-4122-925e-ffa0f131701e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:44.553280 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7adbe-e59a-4122-925e-ffa0f131701e None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:44.553912 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e7adbe-e59a-4122-925e-ffa0f131701e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:44.555348 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e7adbe-e59a-4122-925e-ffa0f131701e None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:44.580005 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 560/1138] 10.222.0.43 () {60 vars in 1132 bytes} [Tue Sep 6 16:04:44 2022] POST /identity/v3/users => generated 286 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:04:46.496896 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949fd949-9b70-4856-a119-f730d9c2a468 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:46.497468 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949fd949-9b70-4856-a119-f730d9c2a468 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:46.497468 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949fd949-9b70-4856-a119-f730d9c2a468 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:46.498781 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 579/1139] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:04:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:04:46.505331 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea086e34-9b1b-4e0d-b605-461f2c1c8498 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:46.506064 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea086e34-9b1b-4e0d-b605-461f2c1c8498 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:46.506230 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea086e34-9b1b-4e0d-b605-461f2c1c8498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:46.534257 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-ea086e34-9b1b-4e0d-b605-461f2c1c8498 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:04:46.535375 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ea086e34-9b1b-4e0d-b605-461f2c1c8498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:04:46.606000 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 561/1140] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:04:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:04:46.870795 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949fd949-9b70-4856-a119-f730d9c2a468 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:46.987144 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949fd949-9b70-4856-a119-f730d9c2a468 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:47.021800 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb918524-77ad-47a5-87a6-b56581d3c2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:47.022415 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb918524-77ad-47a5-87a6-b56581d3c2e6 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:47.022693 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb918524-77ad-47a5-87a6-b56581d3c2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:47.022960 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb918524-77ad-47a5-87a6-b56581d3c2e6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:47.033841 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb918524-77ad-47a5-87a6-b56581d3c2e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:04:47.038640 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-bb918524-77ad-47a5-87a6-b56581d3c2e6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:04:47.038640 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:47.038640 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:47.038640 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:47.038640 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:47.038640 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:47.038640 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:47.038640 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:47.038640 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:47.038640 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:47.038640 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:47.038640 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:47.038640 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:04:47.038640 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:04:47.038640 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:04:47.038640 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:04:47.039690 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:04:47.039690 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:04:47.039690 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:04:47.039690 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:04:47.039690 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:47.039690 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:47.039690 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:47.039690 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:47.039690 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:47.039690 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:47.039690 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:47.039690 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:47.039690 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:47.039690 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:47.039690 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:47.039690 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:47.040807 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:47.040807 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:47.040807 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:47.040807 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:47.040807 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:04:47.040807 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:04:47.040807 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:04:47.040807 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:04:47.040807 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:04:47.040807 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:04:47.040807 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:04:47.040807 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:04:47.040807 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 580/1141] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 6 16:04:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:04:47.046179 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea086e34-9b1b-4e0d-b605-461f2c1c8498 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:47.060985 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:47.146611 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea086e34-9b1b-4e0d-b605-461f2c1c8498 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:47.181970 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f3bcc7-510b-4654-80be-dc64c8fd5ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:47.182921 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f3bcc7-510b-4654-80be-dc64c8fd5ec6 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:47.183196 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f3bcc7-510b-4654-80be-dc64c8fd5ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:47.183465 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f3bcc7-510b-4654-80be-dc64c8fd5ec6 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:47.184132 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f3bcc7-510b-4654-80be-dc64c8fd5ec6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:47.184435 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f3bcc7-510b-4654-80be-dc64c8fd5ec6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:47.185852 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f3bcc7-510b-4654-80be-dc64c8fd5ec6 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:47.194049 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 562/1142] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 6 16:04:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:04:47.200180 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb918524-77ad-47a5-87a6-b56581d3c2e6 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:47.229450 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:47.311094 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb918524-77ad-47a5-87a6-b56581d3c2e6 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:47.349957 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc50d7a4-625f-4804-ae72-059575a65db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:47.350585 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc50d7a4-625f-4804-ae72-059575a65db6 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:47.350857 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc50d7a4-625f-4804-ae72-059575a65db6 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:47.351124 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc50d7a4-625f-4804-ae72-059575a65db6 None admin] PATH_INFO: `/v3/users/7aa7fa78e8a148ce87f69b8ba45b5827` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:47.364086 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc50d7a4-625f-4804-ae72-059575a65db6 None admin] RBAC: Authorizing `identity:get_user(user_id=7aa7fa78e8a148ce87f69b8ba45b5827)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:47.365568 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc50d7a4-625f-4804-ae72-059575a65db6 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:47.367161 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 581/1143] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 6 16:04:47 2022] GET /identity/v3/users/7aa7fa78e8a148ce87f69b8ba45b5827 => generated 286 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:47.373753 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f3bcc7-510b-4654-80be-dc64c8fd5ec6 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:47.449825 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:47.478161 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f3bcc7-510b-4654-80be-dc64c8fd5ec6 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:47.525741 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60645204-05aa-4138-af4e-afaa3337d933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:47.526333 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60645204-05aa-4138-af4e-afaa3337d933 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:47.526602 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60645204-05aa-4138-af4e-afaa3337d933 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:47.526874 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60645204-05aa-4138-af4e-afaa3337d933 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:47.607067 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60645204-05aa-4138-af4e-afaa3337d933 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:47.608529 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60645204-05aa-4138-af4e-afaa3337d933 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:47.681886 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:47.709436 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 563/1144] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 6 16:04:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 16:04:47.715850 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc50d7a4-625f-4804-ae72-059575a65db6 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:47.806402 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:47.825393 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc50d7a4-625f-4804-ae72-059575a65db6 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:47.861527 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a031b01-8190-4814-9bae-b84fee26e6c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:47.862137 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a031b01-8190-4814-9bae-b84fee26e6c6 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:47.862361 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a031b01-8190-4814-9bae-b84fee26e6c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:47.862664 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a031b01-8190-4814-9bae-b84fee26e6c6 None admin] PATH_INFO: `/v3/projects/b91954a6ae8646049f0d79b10fa5862e` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:47.863880 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a031b01-8190-4814-9bae-b84fee26e6c6 None admin] RBAC: Authorizing `identity:get_project(project_id=b91954a6ae8646049f0d79b10fa5862e)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:47.865336 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a031b01-8190-4814-9bae-b84fee26e6c6 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:47.866972 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 582/1145] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 6 16:04:47 2022] GET /identity/v3/projects/b91954a6ae8646049f0d79b10fa5862e => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:47.874215 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60645204-05aa-4138-af4e-afaa3337d933 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:47.977636 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60645204-05aa-4138-af4e-afaa3337d933 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:48.012779 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8f1125-8394-4412-8094-15a734fda6c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:48.013369 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8f1125-8394-4412-8094-15a734fda6c5 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:48.013624 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8f1125-8394-4412-8094-15a734fda6c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:48.013913 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8f1125-8394-4412-8094-15a734fda6c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:48.014631 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8f1125-8394-4412-8094-15a734fda6c5 None admin] RBAC: Adding query filter params (role.id=edaabdea8be84a7980fb2c651fe52f63, scope.project.id=b91954a6ae8646049f0d79b10fa5862e, user.id=7aa7fa78e8a148ce87f69b8ba45b5827) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:48.014933 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8f1125-8394-4412-8094-15a734fda6c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:48.016394 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8f1125-8394-4412-8094-15a734fda6c5 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:48.021847 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 564/1146] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 6 16:04:47 2022] GET /identity/v3/role_assignments?user.id=7aa7fa78e8a148ce87f69b8ba45b5827&scope.project.id=b91954a6ae8646049f0d79b10fa5862e&role.id=edaabdea8be84a7980fb2c651fe52f63 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:49.865387 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e6fc99-46bd-4359-90dc-bed0a71cfe91 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:49.866270 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e6fc99-46bd-4359-90dc-bed0a71cfe91 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:49.866270 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e6fc99-46bd-4359-90dc-bed0a71cfe91 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:49.868390 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 583/1147] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:04:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:04:49.873769 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efa245c-7f95-405e-b93c-0ef83e0ab29f None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:49.874185 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efa245c-7f95-405e-b93c-0ef83e0ab29f None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:49.874273 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efa245c-7f95-405e-b93c-0ef83e0ab29f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:49.894150 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-7efa245c-7f95-405e-b93c-0ef83e0ab29f None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:04:49.895323 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-7efa245c-7f95-405e-b93c-0ef83e0ab29f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:04:49.925475 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:49.965910 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 565/1148] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:04:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:04:50.240873 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e6fc99-46bd-4359-90dc-bed0a71cfe91 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:50.351786 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e6fc99-46bd-4359-90dc-bed0a71cfe91 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:50.387422 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ef9507-2db4-4be8-899b-012a54c8ecad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:50.388043 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ef9507-2db4-4be8-899b-012a54c8ecad None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:50.388324 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ef9507-2db4-4be8-899b-012a54c8ecad None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:50.388591 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ef9507-2db4-4be8-899b-012a54c8ecad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:50.399857 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ef9507-2db4-4be8-899b-012a54c8ecad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:04:50.404635 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-17ef9507-2db4-4be8-899b-012a54c8ecad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:04:50.404635 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:50.404635 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:50.404635 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:50.404635 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:50.404635 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:50.404635 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:50.404635 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:50.404635 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:50.404635 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:50.404635 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:50.404635 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:50.404635 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:04:50.404635 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:04:50.404635 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:04:50.404635 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:04:50.405824 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:04:50.405824 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:04:50.405824 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:04:50.405824 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:04:50.405824 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:50.405824 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:50.405824 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:50.405824 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:50.405824 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:50.405824 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:50.405824 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:50.405824 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:50.405824 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:50.405824 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:50.405824 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:50.405824 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:50.406833 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:50.406833 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:50.406833 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:50.406833 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:50.406833 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:04:50.406833 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:04:50.406833 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:04:50.406833 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:04:50.406833 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:04:50.406833 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:04:50.406833 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:04:50.406833 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:04:50.408935 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 584/1149] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 6 16:04:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:04:50.412514 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efa245c-7f95-405e-b93c-0ef83e0ab29f None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:50.524607 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efa245c-7f95-405e-b93c-0ef83e0ab29f None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:50.567960 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e6fb17-0e60-41ce-8298-34efa603fb5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:50.568559 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e6fb17-0e60-41ce-8298-34efa603fb5e None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:50.568861 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e6fb17-0e60-41ce-8298-34efa603fb5e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:50.569136 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e6fb17-0e60-41ce-8298-34efa603fb5e None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:50.569803 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e6fb17-0e60-41ce-8298-34efa603fb5e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:50.570100 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e6fb17-0e60-41ce-8298-34efa603fb5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:50.571484 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e6fb17-0e60-41ce-8298-34efa603fb5e None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:50.580118 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 566/1150] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 6 16:04:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:50.586508 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ef9507-2db4-4be8-899b-012a54c8ecad None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:50.611693 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:50.698868 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ef9507-2db4-4be8-899b-012a54c8ecad None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:50.734031 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465b3b91-4d10-4e0c-8a77-749b233321a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:50.734638 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b3b91-4d10-4e0c-8a77-749b233321a1 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:50.734904 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b3b91-4d10-4e0c-8a77-749b233321a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:50.735166 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b3b91-4d10-4e0c-8a77-749b233321a1 None admin] PATH_INFO: `/v3/users/7aa7fa78e8a148ce87f69b8ba45b5827` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:50.736395 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465b3b91-4d10-4e0c-8a77-749b233321a1 None admin] RBAC: Authorizing `identity:get_user(user_id=7aa7fa78e8a148ce87f69b8ba45b5827)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:50.737846 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465b3b91-4d10-4e0c-8a77-749b233321a1 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:50.739391 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 585/1151] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 6 16:04:50 2022] GET /identity/v3/users/7aa7fa78e8a148ce87f69b8ba45b5827 => generated 286 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:50.748672 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e6fb17-0e60-41ce-8298-34efa603fb5e None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:50.782950 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:50.865186 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e6fb17-0e60-41ce-8298-34efa603fb5e None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:50.899373 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba323a0a-def9-4672-8ff7-4bfd302fcbfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:50.899956 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba323a0a-def9-4672-8ff7-4bfd302fcbfd None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:50.900237 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba323a0a-def9-4672-8ff7-4bfd302fcbfd None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:50.900504 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba323a0a-def9-4672-8ff7-4bfd302fcbfd None admin] PATH_INFO: `/v3/projects/b91954a6ae8646049f0d79b10fa5862e` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:50.901668 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba323a0a-def9-4672-8ff7-4bfd302fcbfd None admin] RBAC: Authorizing `identity:get_project(project_id=b91954a6ae8646049f0d79b10fa5862e)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:50.903102 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba323a0a-def9-4672-8ff7-4bfd302fcbfd None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:50.904718 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 567/1152] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 6 16:04:50 2022] GET /identity/v3/projects/b91954a6ae8646049f0d79b10fa5862e => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:50.913379 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465b3b91-4d10-4e0c-8a77-749b233321a1 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:50.999801 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:51.032662 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465b3b91-4d10-4e0c-8a77-749b233321a1 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:51.068215 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f6f66a-5ad8-4b32-9d0b-6cb34dfd0272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:51.068801 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f6f66a-5ad8-4b32-9d0b-6cb34dfd0272 None admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:51.069093 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f6f66a-5ad8-4b32-9d0b-6cb34dfd0272 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:51.069367 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f6f66a-5ad8-4b32-9d0b-6cb34dfd0272 None admin] PATH_INFO: `/v3/projects/b91954a6ae8646049f0d79b10fa5862e/users/7aa7fa78e8a148ce87f69b8ba45b5827/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:51.071488 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f6f66a-5ad8-4b32-9d0b-6cb34dfd0272 None admin] RBAC: Authorizing `identity:create_grant(project_id=b91954a6ae8646049f0d79b10fa5862e, user_id=7aa7fa78e8a148ce87f69b8ba45b5827, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:51.072964 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f6f66a-5ad8-4b32-9d0b-6cb34dfd0272 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:51.088142 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 586/1153] 10.222.0.43 () {58 vars in 1439 bytes} [Tue Sep 6 16:04:50 2022] PUT /identity/v3/projects/b91954a6ae8646049f0d79b10fa5862e/users/7aa7fa78e8a148ce87f69b8ba45b5827/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:04:52.964033 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec902f7-1497-41b8-b676-4a8738b74b2e None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:52.964930 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec902f7-1497-41b8-b676-4a8738b74b2e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:52.964930 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec902f7-1497-41b8-b676-4a8738b74b2e None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:52.967071 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 568/1154] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:04:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:04:52.974032 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dcbce2-1b53-46ef-925e-e4aadb58aa79 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:52.974470 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dcbce2-1b53-46ef-925e-e4aadb58aa79 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:52.974627 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dcbce2-1b53-46ef-925e-e4aadb58aa79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:53.000288 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-61dcbce2-1b53-46ef-925e-e4aadb58aa79 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:04:53.001517 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-61dcbce2-1b53-46ef-925e-e4aadb58aa79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:04:53.072516 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 587/1155] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:04:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:04:53.338370 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec902f7-1497-41b8-b676-4a8738b74b2e None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:53.444917 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:53.450248 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec902f7-1497-41b8-b676-4a8738b74b2e None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:53.483630 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4820d72a-c105-4fb9-93cf-9ea26a74e702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:53.484293 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4820d72a-c105-4fb9-93cf-9ea26a74e702 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:53.484569 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4820d72a-c105-4fb9-93cf-9ea26a74e702 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:53.484881 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4820d72a-c105-4fb9-93cf-9ea26a74e702 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:53.495225 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4820d72a-c105-4fb9-93cf-9ea26a74e702 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:04:53.500044 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-4820d72a-c105-4fb9-93cf-9ea26a74e702 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:04:53.500044 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:53.500044 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:53.500044 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:53.500044 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:53.500044 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:53.500044 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:53.500044 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:53.500044 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:53.500044 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:53.500044 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:53.500044 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:53.500044 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:04:53.500044 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:04:53.500044 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:04:53.500044 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:04:53.501095 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:04:53.501095 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:04:53.501095 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:04:53.501095 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:04:53.501095 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:53.501095 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:53.501095 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:53.501095 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:53.501095 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:53.501095 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:53.501095 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:53.501095 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:53.501095 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:53.501095 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:53.501095 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:53.501095 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:53.502041 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:53.502041 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:53.502041 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:53.502041 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:53.502041 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:04:53.502041 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:04:53.502041 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:04:53.502041 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:04:53.502041 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:04:53.502041 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:04:53.502041 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:04:53.502041 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:04:53.502041 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 569/1156] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 6 16:04:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:04:53.508141 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dcbce2-1b53-46ef-925e-e4aadb58aa79 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:53.596516 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:53.631366 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dcbce2-1b53-46ef-925e-e4aadb58aa79 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:53.674841 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f551d41f-86fa-4111-adf5-da55dc723f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:53.675446 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f551d41f-86fa-4111-adf5-da55dc723f3f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:53.675721 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f551d41f-86fa-4111-adf5-da55dc723f3f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:53.675990 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f551d41f-86fa-4111-adf5-da55dc723f3f None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:53.676654 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f551d41f-86fa-4111-adf5-da55dc723f3f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:53.676980 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f551d41f-86fa-4111-adf5-da55dc723f3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:53.678359 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f551d41f-86fa-4111-adf5-da55dc723f3f None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:53.686456 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 588/1157] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 6 16:04:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:53.692593 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4820d72a-c105-4fb9-93cf-9ea26a74e702 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:53.801336 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4820d72a-c105-4fb9-93cf-9ea26a74e702 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:53.841086 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe5d438-2129-4759-9825-e94d0dc01ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:53.841692 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe5d438-2129-4759-9825-e94d0dc01ec0 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:53.841963 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe5d438-2129-4759-9825-e94d0dc01ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:53.842226 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe5d438-2129-4759-9825-e94d0dc01ec0 None admin] PATH_INFO: `/v3/users/7aa7fa78e8a148ce87f69b8ba45b5827` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:53.843495 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe5d438-2129-4759-9825-e94d0dc01ec0 None admin] RBAC: Authorizing `identity:get_user(user_id=7aa7fa78e8a148ce87f69b8ba45b5827)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:53.844961 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe5d438-2129-4759-9825-e94d0dc01ec0 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:53.846613 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 570/1158] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 6 16:04:53 2022] GET /identity/v3/users/7aa7fa78e8a148ce87f69b8ba45b5827 => generated 286 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:53.855506 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f551d41f-86fa-4111-adf5-da55dc723f3f None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:53.962343 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f551d41f-86fa-4111-adf5-da55dc723f3f None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:53.992287 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:54.001181 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2c8292-7bc1-4e9f-8a17-7e2dad09abdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:54.001776 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c8292-7bc1-4e9f-8a17-7e2dad09abdb None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:54.002048 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c8292-7bc1-4e9f-8a17-7e2dad09abdb None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:54.002308 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c8292-7bc1-4e9f-8a17-7e2dad09abdb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:54.071307 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2c8292-7bc1-4e9f-8a17-7e2dad09abdb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:54.072800 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2c8292-7bc1-4e9f-8a17-7e2dad09abdb None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:54.170492 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 589/1159] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 6 16:04:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 16:04:54.179164 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe5d438-2129-4759-9825-e94d0dc01ec0 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:54.218986 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:54.294811 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe5d438-2129-4759-9825-e94d0dc01ec0 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:54.330198 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788e4408-518b-40c9-9f6c-46db191deeda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:54.330816 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e4408-518b-40c9-9f6c-46db191deeda None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:54.331085 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e4408-518b-40c9-9f6c-46db191deeda None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:54.331350 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e4408-518b-40c9-9f6c-46db191deeda None admin] PATH_INFO: `/v3/projects/b91954a6ae8646049f0d79b10fa5862e` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:54.332575 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788e4408-518b-40c9-9f6c-46db191deeda None admin] RBAC: Authorizing `identity:get_project(project_id=b91954a6ae8646049f0d79b10fa5862e)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:54.334056 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788e4408-518b-40c9-9f6c-46db191deeda None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:54.336063 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 571/1160] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 6 16:04:54 2022] GET /identity/v3/projects/b91954a6ae8646049f0d79b10fa5862e => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:54.344894 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2c8292-7bc1-4e9f-8a17-7e2dad09abdb None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:54.376880 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:54.445792 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2c8292-7bc1-4e9f-8a17-7e2dad09abdb None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:54.480228 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bf8bb7-7282-4f15-9b0b-a6fc88dac6c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:54.480844 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bf8bb7-7282-4f15-9b0b-a6fc88dac6c3 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:54.481119 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bf8bb7-7282-4f15-9b0b-a6fc88dac6c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:54.481384 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bf8bb7-7282-4f15-9b0b-a6fc88dac6c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:54.482093 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bf8bb7-7282-4f15-9b0b-a6fc88dac6c3 None admin] RBAC: Adding query filter params (role.id=edaabdea8be84a7980fb2c651fe52f63, scope.project.id=b91954a6ae8646049f0d79b10fa5862e, user.id=7aa7fa78e8a148ce87f69b8ba45b5827) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:54.482404 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bf8bb7-7282-4f15-9b0b-a6fc88dac6c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:54.483830 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bf8bb7-7282-4f15-9b0b-a6fc88dac6c3 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:54.489562 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 590/1161] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 6 16:04:54 2022] GET /identity/v3/role_assignments?user.id=7aa7fa78e8a148ce87f69b8ba45b5827&scope.project.id=b91954a6ae8646049f0d79b10fa5862e&role.id=edaabdea8be84a7980fb2c651fe52f63 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:04:56.418640 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89386991-085a-416d-8c8a-8bb2aaa5aaf9 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:56.419194 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89386991-085a-416d-8c8a-8bb2aaa5aaf9 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:56.419194 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89386991-085a-416d-8c8a-8bb2aaa5aaf9 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:56.420553 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 572/1162] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:04:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 16:04:56.427099 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e88e3-1de9-4937-9f4c-8228952d642f None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:56.427807 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e88e3-1de9-4937-9f4c-8228952d642f None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:56.427964 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e88e3-1de9-4937-9f4c-8228952d642f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:56.457132 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-0a7e88e3-1de9-4937-9f4c-8228952d642f None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:04:56.458324 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0a7e88e3-1de9-4937-9f4c-8228952d642f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:04:56.527304 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 591/1163] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:04:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:04:56.812961 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89386991-085a-416d-8c8a-8bb2aaa5aaf9 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:56.917872 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:56.929112 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89386991-085a-416d-8c8a-8bb2aaa5aaf9 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:56.967547 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9128f459-4cb6-4834-b5bb-abf1aa5c6a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:56.968203 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9128f459-4cb6-4834-b5bb-abf1aa5c6a7d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:56.968475 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9128f459-4cb6-4834-b5bb-abf1aa5c6a7d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:56.968743 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9128f459-4cb6-4834-b5bb-abf1aa5c6a7d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:57.040380 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9128f459-4cb6-4834-b5bb-abf1aa5c6a7d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:57.041868 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9128f459-4cb6-4834-b5bb-abf1aa5c6a7d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:57.144466 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:57.152533 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 573/1164] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 6 16:04:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 16:04:57.158933 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7e88e3-1de9-4937-9f4c-8228952d642f None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:57.189001 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:57.266483 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7e88e3-1de9-4937-9f4c-8228952d642f None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:57.301707 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aa9ffe-498a-43e0-bfb5-3bbd12c17eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:57.302326 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa9ffe-498a-43e0-bfb5-3bbd12c17eaa None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:57.302599 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa9ffe-498a-43e0-bfb5-3bbd12c17eaa None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:57.302863 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa9ffe-498a-43e0-bfb5-3bbd12c17eaa None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:57.309799 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aa9ffe-498a-43e0-bfb5-3bbd12c17eaa None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:57.311259 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aa9ffe-498a-43e0-bfb5-3bbd12c17eaa None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:57.322691 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-18aa9ffe-498a-43e0-bfb5-3bbd12c17eaa None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 06 16:04:57.322691 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:57.322691 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 06 16:04:57.322691 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 06 16:04:57.322691 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:04:57.322691 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:04:57.322691 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:04:57.322691 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:04:57.322691 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 06 16:04:57.322691 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:04:57.322691 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 16:04:57.322691 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:04:57.322691 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:57.322691 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:57.322691 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:57.322691 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:57.322691 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:57.323860 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:57.323860 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:57.323860 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:57.323860 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:57.323860 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:57.323860 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:57.323860 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:04:57.323860 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:04:57.323860 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:04:57.323860 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:04:57.323860 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:57.323860 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:57.323860 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:57.323860 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:57.323860 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:57.323860 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:57.324892 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:57.324892 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:57.324892 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:57.324892 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:57.324892 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:57.324892 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:57.324892 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:57.324892 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:57.324892 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:57.324892 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:57.324892 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 06 16:04:57.324892 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:04:57.324892 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 06 16:04:57.324892 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:04:57.324892 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 592/1165] 10.222.0.43 () {56 vars in 1117 bytes} [Tue Sep 6 16:04:57 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:04:57.331432 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9128f459-4cb6-4834-b5bb-abf1aa5c6a7d None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:57.447785 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9128f459-4cb6-4834-b5bb-abf1aa5c6a7d None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:57.484635 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7e0fd1-0a7d-451d-9edc-f518ae612cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:57.485264 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7e0fd1-0a7d-451d-9edc-f518ae612cc9 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:57.485549 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7e0fd1-0a7d-451d-9edc-f518ae612cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:57.485822 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7e0fd1-0a7d-451d-9edc-f518ae612cc9 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:57.486470 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7e0fd1-0a7d-451d-9edc-f518ae612cc9 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:57.486771 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7e0fd1-0a7d-451d-9edc-f518ae612cc9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:57.488142 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7e0fd1-0a7d-451d-9edc-f518ae612cc9 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:57.495261 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 574/1166] 10.222.0.43 () {56 vars in 1115 bytes} [Tue Sep 6 16:04:57 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:59.356027 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b99ed9c-a6ea-4032-824a-14308954c063 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:59.357101 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b99ed9c-a6ea-4032-824a-14308954c063 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:59.357101 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b99ed9c-a6ea-4032-824a-14308954c063 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:59.359070 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 593/1167] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:04:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:04:59.364308 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01b754f-6eb3-47d4-bac0-7b1c0e748983 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:59.364743 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01b754f-6eb3-47d4-bac0-7b1c0e748983 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:59.364897 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01b754f-6eb3-47d4-bac0-7b1c0e748983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:59.384861 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-e01b754f-6eb3-47d4-bac0-7b1c0e748983 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:04:59.386040 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e01b754f-6eb3-47d4-bac0-7b1c0e748983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:04:59.439984 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:59.455071 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 575/1168] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:04:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:04:59.719829 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b99ed9c-a6ea-4032-824a-14308954c063 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:59.821820 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:59.846345 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b99ed9c-a6ea-4032-824a-14308954c063 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:59.881984 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efd2c1a-5c13-437e-b363-5ed07c2fc0a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:59.882567 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd2c1a-5c13-437e-b363-5ed07c2fc0a7 None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:59.882839 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd2c1a-5c13-437e-b363-5ed07c2fc0a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:59.883103 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd2c1a-5c13-437e-b363-5ed07c2fc0a7 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:59.883751 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efd2c1a-5c13-437e-b363-5ed07c2fc0a7 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:59.885139 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efd2c1a-5c13-437e-b363-5ed07c2fc0a7 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:59.907277 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 594/1169] 10.222.0.43 () {60 vars in 1138 bytes} [Tue Sep 6 16:04:59 2022] POST /identity/v3/domains => generated 267 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:05:01.795374 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1115ca7-2805-4811-a40e-aad241e6e1eb None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:05:01.796027 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1115ca7-2805-4811-a40e-aad241e6e1eb None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:05:01.796027 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1115ca7-2805-4811-a40e-aad241e6e1eb None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:05:01.797364 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 576/1170] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:05:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:05:01.803711 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf477c-47b7-4459-9c32-a92619898722 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:05:01.804432 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf477c-47b7-4459-9c32-a92619898722 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:05:01.804625 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf477c-47b7-4459-9c32-a92619898722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:05:01.832988 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-bedf477c-47b7-4459-9c32-a92619898722 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:05:01.834184 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-bedf477c-47b7-4459-9c32-a92619898722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:05:01.903543 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 595/1171] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:05:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:05:02.179594 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1115ca7-2805-4811-a40e-aad241e6e1eb None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:05:02.294631 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1115ca7-2805-4811-a40e-aad241e6e1eb None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:05:02.332431 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77c59f0-9ab2-4ad0-bb6d-6efe60f00888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:05:02.333108 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77c59f0-9ab2-4ad0-bb6d-6efe60f00888 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:05:02.333365 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77c59f0-9ab2-4ad0-bb6d-6efe60f00888 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:05:02.333638 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77c59f0-9ab2-4ad0-bb6d-6efe60f00888 None admin] PATH_INFO: `/v3/domains/e4713857d8c4419e8b59643e494fa07c` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:05:02.344611 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77c59f0-9ab2-4ad0-bb6d-6efe60f00888 None admin] RBAC: Authorizing `identity:get_domain(domain_id=e4713857d8c4419e8b59643e494fa07c)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:05:02.346117 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77c59f0-9ab2-4ad0-bb6d-6efe60f00888 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:05:02.347709 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 577/1172] 10.222.0.43 () {56 vars in 1183 bytes} [Tue Sep 6 16:05:02 2022] GET /identity/v3/domains/e4713857d8c4419e8b59643e494fa07c => generated 267 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:05:02.356736 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedf477c-47b7-4459-9c32-a92619898722 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:05:02.433241 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:05:02.459088 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedf477c-47b7-4459-9c32-a92619898722 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:05:02.494476 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e689bf2-a584-4865-a13d-9a8ea35fc588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:05:02.495086 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e689bf2-a584-4865-a13d-9a8ea35fc588 None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:05:02.495359 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e689bf2-a584-4865-a13d-9a8ea35fc588 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:05:02.495631 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e689bf2-a584-4865-a13d-9a8ea35fc588 None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:05:02.496287 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e689bf2-a584-4865-a13d-9a8ea35fc588 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:05:02.497762 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e689bf2-a584-4865-a13d-9a8ea35fc588 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:05:02.529190 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 596/1173] 10.222.0.43 () {60 vars in 1141 bytes} [Tue Sep 6 16:05:02 2022] POST /identity/v3/projects => generated 361 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:05:04.444944 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e161352-3e26-4661-8808-078967b0eadf None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:05:04.445529 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e161352-3e26-4661-8808-078967b0eadf None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:05:04.445529 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e161352-3e26-4661-8808-078967b0eadf None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:05:04.446900 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 578/1174] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:05:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:05:04.454054 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31d772a-5ab3-4518-8729-37966ef3f328 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:05:04.454840 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31d772a-5ab3-4518-8729-37966ef3f328 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:05:04.455009 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31d772a-5ab3-4518-8729-37966ef3f328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:05:04.479905 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-f31d772a-5ab3-4518-8729-37966ef3f328 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:05:04.481138 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f31d772a-5ab3-4518-8729-37966ef3f328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:05:04.552018 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 597/1175] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:05:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:05:04.822569 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e161352-3e26-4661-8808-078967b0eadf None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:05:04.881057 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:05:04.937135 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e161352-3e26-4661-8808-078967b0eadf None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:05:04.976010 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538a20c2-4378-46b5-9854-acd573bb6206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:05:04.976629 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538a20c2-4378-46b5-9854-acd573bb6206 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:05:04.976937 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538a20c2-4378-46b5-9854-acd573bb6206 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:05:04.977211 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538a20c2-4378-46b5-9854-acd573bb6206 None admin] PATH_INFO: `/v3/domains/e4713857d8c4419e8b59643e494fa07c` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:05:04.978460 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538a20c2-4378-46b5-9854-acd573bb6206 None admin] RBAC: Authorizing `identity:get_domain(domain_id=e4713857d8c4419e8b59643e494fa07c)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:05:04.979915 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538a20c2-4378-46b5-9854-acd573bb6206 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:05:04.981554 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 579/1176] 10.222.0.43 () {56 vars in 1183 bytes} [Tue Sep 6 16:05:04 2022] GET /identity/v3/domains/e4713857d8c4419e8b59643e494fa07c => generated 267 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:05:04.988143 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31d772a-5ab3-4518-8729-37966ef3f328 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:05:05.050539 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:05:05.095184 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31d772a-5ab3-4518-8729-37966ef3f328 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:05:05.129958 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d487e5b6-a9c9-45e7-962c-847852429ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:05:05.130547 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d487e5b6-a9c9-45e7-962c-847852429ad3 None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:05:05.130820 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d487e5b6-a9c9-45e7-962c-847852429ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:05:05.131080 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d487e5b6-a9c9-45e7-962c-847852429ad3 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:05:05.131710 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d487e5b6-a9c9-45e7-962c-847852429ad3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:05:05.133159 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d487e5b6-a9c9-45e7-962c-847852429ad3 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:05:05.157371 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 598/1177] 10.222.0.43 () {60 vars in 1132 bytes} [Tue Sep 6 16:05:04 2022] POST /identity/v3/users => generated 311 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:05:07.100242 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c81c0de-d0a6-41f1-ba82-fa6630b7fe63 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:05:07.101177 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c81c0de-d0a6-41f1-ba82-fa6630b7fe63 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:05:07.101177 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c81c0de-d0a6-41f1-ba82-fa6630b7fe63 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:05:07.103288 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 580/1178] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:05:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:05:07.109427 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca1c8a-f3c2-4913-9a3c-e054ca4fa67a None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:05:07.110001 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca1c8a-f3c2-4913-9a3c-e054ca4fa67a None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:05:07.114392 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca1c8a-f3c2-4913-9a3c-e054ca4fa67a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:05:07.141398 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-11ca1c8a-f3c2-4913-9a3c-e054ca4fa67a None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:05:07.142585 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-11ca1c8a-f3c2-4913-9a3c-e054ca4fa67a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:05:07.215328 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 599/1179] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:05:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:05:07.492747 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c81c0de-d0a6-41f1-ba82-fa6630b7fe63 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:05:07.623223 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c81c0de-d0a6-41f1-ba82-fa6630b7fe63 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:05:07.641444 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:05:07.665936 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace5f38e-d331-461e-9fe4-f9f5a37ccf7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:05:07.666585 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace5f38e-d331-461e-9fe4-f9f5a37ccf7c None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:05:07.666857 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace5f38e-d331-461e-9fe4-f9f5a37ccf7c None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:05:07.667114 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace5f38e-d331-461e-9fe4-f9f5a37ccf7c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:05:07.680837 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace5f38e-d331-461e-9fe4-f9f5a37ccf7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:05:07.685903 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-ace5f38e-d331-461e-9fe4-f9f5a37ccf7c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:05:07.685903 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:05:07.685903 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:05:07.685903 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:05:07.685903 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:05:07.685903 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:05:07.685903 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:05:07.685903 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:05:07.685903 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:05:07.685903 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:05:07.685903 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:05:07.685903 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:05:07.685903 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:05:07.685903 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:05:07.685903 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:05:07.685903 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:05:07.687199 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:05:07.687199 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:05:07.687199 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:05:07.687199 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:05:07.687199 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:05:07.687199 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:05:07.687199 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:05:07.687199 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:05:07.687199 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:05:07.687199 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:05:07.687199 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:05:07.687199 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:05:07.687199 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:05:07.687199 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:05:07.687199 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:05:07.687199 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:05:07.688343 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:05:07.688343 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:05:07.688343 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:05:07.688343 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:05:07.688343 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:05:07.688343 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:05:07.688343 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:05:07.688343 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:05:07.688343 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:05:07.688343 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:05:07.688343 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:05:07.688343 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:05:07.688343 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 581/1180] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 6 16:05:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:05:07.694723 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ca1c8a-f3c2-4913-9a3c-e054ca4fa67a None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:05:07.745838 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:05:07.802183 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ca1c8a-f3c2-4913-9a3c-e054ca4fa67a None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:05:07.838265 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a11a476-359d-4b9f-bb8d-c5b7e1d16964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:05:07.838866 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a11a476-359d-4b9f-bb8d-c5b7e1d16964 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:05:07.839145 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a11a476-359d-4b9f-bb8d-c5b7e1d16964 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:05:07.839423 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a11a476-359d-4b9f-bb8d-c5b7e1d16964 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:05:07.840085 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a11a476-359d-4b9f-bb8d-c5b7e1d16964 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:05:07.840382 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a11a476-359d-4b9f-bb8d-c5b7e1d16964 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:05:07.841798 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a11a476-359d-4b9f-bb8d-c5b7e1d16964 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:05:07.850073 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 600/1181] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 6 16:05:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:05:07.856314 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace5f38e-d331-461e-9fe4-f9f5a37ccf7c None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:05:07.958337 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace5f38e-d331-461e-9fe4-f9f5a37ccf7c None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:05:07.993099 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eb73c0-f80e-4861-a9b5-650b48b07e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:05:07.993717 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb73c0-f80e-4861-a9b5-650b48b07e46 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:05:07.993993 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb73c0-f80e-4861-a9b5-650b48b07e46 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:05:07.994267 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb73c0-f80e-4861-a9b5-650b48b07e46 None admin] PATH_INFO: `/v3/users/46a4863eb67344a7a479fd3f78ae8927` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:05:08.008151 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eb73c0-f80e-4861-a9b5-650b48b07e46 None admin] RBAC: Authorizing `identity:get_user(user_id=46a4863eb67344a7a479fd3f78ae8927)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:05:08.009593 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eb73c0-f80e-4861-a9b5-650b48b07e46 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:05:08.011228 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 582/1182] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 6 16:05:07 2022] GET /identity/v3/users/46a4863eb67344a7a479fd3f78ae8927 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:05:08.017755 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a11a476-359d-4b9f-bb8d-c5b7e1d16964 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:05:08.125426 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a11a476-359d-4b9f-bb8d-c5b7e1d16964 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:05:08.132914 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:05:08.162507 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7092257b-e92b-4f80-887c-abd6bc9242ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:05:08.163108 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7092257b-e92b-4f80-887c-abd6bc9242ce None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:05:08.163387 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7092257b-e92b-4f80-887c-abd6bc9242ce None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:05:08.163652 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7092257b-e92b-4f80-887c-abd6bc9242ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:05:08.231580 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7092257b-e92b-4f80-887c-abd6bc9242ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:05:08.233078 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7092257b-e92b-4f80-887c-abd6bc9242ce None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:05:08.335486 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 601/1183] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 6 16:05:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 16:05:08.341922 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eb73c0-f80e-4861-a9b5-650b48b07e46 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:05:08.379848 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:05:08.448021 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eb73c0-f80e-4861-a9b5-650b48b07e46 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:05:08.484910 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab904741-2e41-4867-b3a6-e8d0064eba74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:05:08.485540 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab904741-2e41-4867-b3a6-e8d0064eba74 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:05:08.485817 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab904741-2e41-4867-b3a6-e8d0064eba74 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:05:08.486091 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab904741-2e41-4867-b3a6-e8d0064eba74 None admin] PATH_INFO: `/v3/projects/972c629ada64475ba58c29c585d058dc` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:05:08.487256 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab904741-2e41-4867-b3a6-e8d0064eba74 None admin] RBAC: Authorizing `identity:get_project(project_id=972c629ada64475ba58c29c585d058dc)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:05:08.488715 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab904741-2e41-4867-b3a6-e8d0064eba74 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:05:08.490366 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 583/1184] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 6 16:05:08 2022] GET /identity/v3/projects/972c629ada64475ba58c29c585d058dc => generated 361 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:05:08.497163 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7092257b-e92b-4f80-887c-abd6bc9242ce None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:05:08.507266 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:05:08.603647 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7092257b-e92b-4f80-887c-abd6bc9242ce None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:05:08.638879 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c017c1-9d9f-45bf-96a8-6142008ad01d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:05:08.639541 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c017c1-9d9f-45bf-96a8-6142008ad01d None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:05:08.639819 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c017c1-9d9f-45bf-96a8-6142008ad01d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:05:08.640090 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c017c1-9d9f-45bf-96a8-6142008ad01d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:05:08.640846 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c017c1-9d9f-45bf-96a8-6142008ad01d None admin] RBAC: Adding query filter params (role.id=edaabdea8be84a7980fb2c651fe52f63, scope.project.id=972c629ada64475ba58c29c585d058dc, user.id=46a4863eb67344a7a479fd3f78ae8927) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:05:08.641149 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c017c1-9d9f-45bf-96a8-6142008ad01d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:05:08.642586 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c017c1-9d9f-45bf-96a8-6142008ad01d None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:05:08.648303 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 602/1185] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 6 16:05:08 2022] GET /identity/v3/role_assignments?user.id=46a4863eb67344a7a479fd3f78ae8927&scope.project.id=972c629ada64475ba58c29c585d058dc&role.id=edaabdea8be84a7980fb2c651fe52f63 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:05:10.527989 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c56e1-3589-4186-9b6b-57ad3ce2717b None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:05:10.528538 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c56e1-3589-4186-9b6b-57ad3ce2717b None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:05:10.528538 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c56e1-3589-4186-9b6b-57ad3ce2717b None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:05:10.529939 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 584/1186] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:05:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:05:10.536345 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d071ac77-cffe-454f-a5c5-e93582ca35b1 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:05:10.537077 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d071ac77-cffe-454f-a5c5-e93582ca35b1 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:05:10.537255 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d071ac77-cffe-454f-a5c5-e93582ca35b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:05:10.565288 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-d071ac77-cffe-454f-a5c5-e93582ca35b1 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:05:10.566447 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d071ac77-cffe-454f-a5c5-e93582ca35b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:05:10.631996 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:05:10.640057 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 603/1187] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:05:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:05:10.905308 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74c56e1-3589-4186-9b6b-57ad3ce2717b None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:05:11.031262 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74c56e1-3589-4186-9b6b-57ad3ce2717b None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:05:11.032828 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:05:11.069167 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af963929-f414-4be7-ab47-44814cb6fa23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:05:11.069777 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af963929-f414-4be7-ab47-44814cb6fa23 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:05:11.070050 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af963929-f414-4be7-ab47-44814cb6fa23 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:05:11.070319 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af963929-f414-4be7-ab47-44814cb6fa23 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:05:11.081854 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af963929-f414-4be7-ab47-44814cb6fa23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:05:11.086662 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-af963929-f414-4be7-ab47-44814cb6fa23 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:05:11.086662 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:05:11.086662 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:05:11.086662 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:05:11.086662 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:05:11.086662 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:05:11.086662 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:05:11.086662 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:05:11.086662 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:05:11.086662 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:05:11.086662 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:05:11.086662 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:05:11.086662 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:05:11.086662 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:05:11.086662 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:05:11.086662 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:05:11.087687 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:05:11.087687 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:05:11.087687 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:05:11.087687 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:05:11.087687 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:05:11.087687 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:05:11.087687 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:05:11.087687 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:05:11.087687 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:05:11.087687 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:05:11.087687 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:05:11.087687 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:05:11.087687 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:05:11.087687 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:05:11.087687 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:05:11.087687 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:05:11.088696 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:05:11.088696 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:05:11.088696 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:05:11.088696 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:05:11.088696 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:05:11.088696 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:05:11.088696 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:05:11.088696 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:05:11.088696 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:05:11.088696 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:05:11.088696 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:05:11.088696 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:05:11.088696 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 585/1188] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 6 16:05:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:05:11.093966 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d071ac77-cffe-454f-a5c5-e93582ca35b1 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:05:11.210615 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d071ac77-cffe-454f-a5c5-e93582ca35b1 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:05:11.254697 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7e4596-fe6b-43cd-9f71-707c86512326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:05:11.255308 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e4596-fe6b-43cd-9f71-707c86512326 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:05:11.255576 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e4596-fe6b-43cd-9f71-707c86512326 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:05:11.255808 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e4596-fe6b-43cd-9f71-707c86512326 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:05:11.256463 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7e4596-fe6b-43cd-9f71-707c86512326 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:05:11.256788 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7e4596-fe6b-43cd-9f71-707c86512326 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:05:11.258170 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7e4596-fe6b-43cd-9f71-707c86512326 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:05:11.266592 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 604/1189] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 6 16:05:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:05:11.272700 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af963929-f414-4be7-ab47-44814cb6fa23 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:05:11.374219 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af963929-f414-4be7-ab47-44814cb6fa23 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:05:11.409940 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02150e97-e508-4b2a-b34f-bc604be71080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:05:11.410554 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02150e97-e508-4b2a-b34f-bc604be71080 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:05:11.410831 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02150e97-e508-4b2a-b34f-bc604be71080 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:05:11.411102 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02150e97-e508-4b2a-b34f-bc604be71080 None admin] PATH_INFO: `/v3/users/46a4863eb67344a7a479fd3f78ae8927` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:05:11.412267 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02150e97-e508-4b2a-b34f-bc604be71080 None admin] RBAC: Authorizing `identity:get_user(user_id=46a4863eb67344a7a479fd3f78ae8927)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:05:11.413722 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02150e97-e508-4b2a-b34f-bc604be71080 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:05:11.415322 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 586/1190] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 6 16:05:11 2022] GET /identity/v3/users/46a4863eb67344a7a479fd3f78ae8927 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:05:11.423155 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7e4596-fe6b-43cd-9f71-707c86512326 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:05:11.486915 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:05:11.525591 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7e4596-fe6b-43cd-9f71-707c86512326 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:05:11.568073 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fd6451-6806-4d4f-aa76-9b4b00809879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:05:11.568694 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd6451-6806-4d4f-aa76-9b4b00809879 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:05:11.568982 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd6451-6806-4d4f-aa76-9b4b00809879 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:05:11.569250 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd6451-6806-4d4f-aa76-9b4b00809879 None admin] PATH_INFO: `/v3/projects/972c629ada64475ba58c29c585d058dc` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:05:11.570489 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fd6451-6806-4d4f-aa76-9b4b00809879 None admin] RBAC: Authorizing `identity:get_project(project_id=972c629ada64475ba58c29c585d058dc)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:05:11.571918 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fd6451-6806-4d4f-aa76-9b4b00809879 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:05:11.573580 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 605/1191] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 6 16:05:11 2022] GET /identity/v3/projects/972c629ada64475ba58c29c585d058dc => generated 361 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:05:11.580424 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02150e97-e508-4b2a-b34f-bc604be71080 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:05:11.615723 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:05:11.693777 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02150e97-e508-4b2a-b34f-bc604be71080 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:05:11.729823 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc52053-2ada-427e-9540-3b108f6e5a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:05:11.730465 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc52053-2ada-427e-9540-3b108f6e5a0b None admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:05:11.730736 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc52053-2ada-427e-9540-3b108f6e5a0b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:05:11.731001 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc52053-2ada-427e-9540-3b108f6e5a0b None admin] PATH_INFO: `/v3/projects/972c629ada64475ba58c29c585d058dc/users/46a4863eb67344a7a479fd3f78ae8927/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:05:11.733198 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc52053-2ada-427e-9540-3b108f6e5a0b None admin] RBAC: Authorizing `identity:create_grant(project_id=972c629ada64475ba58c29c585d058dc, user_id=46a4863eb67344a7a479fd3f78ae8927, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:05:11.734659 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc52053-2ada-427e-9540-3b108f6e5a0b None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:05:11.752824 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 587/1192] 10.222.0.43 () {58 vars in 1439 bytes} [Tue Sep 6 16:05:11 2022] PUT /identity/v3/projects/972c629ada64475ba58c29c585d058dc/users/46a4863eb67344a7a479fd3f78ae8927/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:05:13.624198 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce5552a-4222-4fbf-8121-d6866ad1b86d None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:05:13.625102 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce5552a-4222-4fbf-8121-d6866ad1b86d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:05:13.625102 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce5552a-4222-4fbf-8121-d6866ad1b86d None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:05:13.627145 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 606/1193] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:05:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:05:13.632582 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed3c0aa-dd85-4239-86ce-a14dcc8883e8 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:05:13.633030 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed3c0aa-dd85-4239-86ce-a14dcc8883e8 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:05:13.633118 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed3c0aa-dd85-4239-86ce-a14dcc8883e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:05:13.654802 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-7ed3c0aa-dd85-4239-86ce-a14dcc8883e8 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:05:13.655992 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-7ed3c0aa-dd85-4239-86ce-a14dcc8883e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:05:13.734942 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 588/1194] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:05:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:05:14.012268 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce5552a-4222-4fbf-8121-d6866ad1b86d None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:05:14.128885 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce5552a-4222-4fbf-8121-d6866ad1b86d None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:05:14.160438 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:05:14.167039 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324f62b6-a6d7-44fc-865a-63659b86d82a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:05:14.167667 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f62b6-a6d7-44fc-865a-63659b86d82a None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:05:14.167941 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f62b6-a6d7-44fc-865a-63659b86d82a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:05:14.168208 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f62b6-a6d7-44fc-865a-63659b86d82a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:05:14.178733 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324f62b6-a6d7-44fc-865a-63659b86d82a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:05:14.183527 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-324f62b6-a6d7-44fc-865a-63659b86d82a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:05:14.183527 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:05:14.183527 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:05:14.183527 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:05:14.183527 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:05:14.183527 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:05:14.183527 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:05:14.183527 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:05:14.183527 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:05:14.183527 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:05:14.183527 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:05:14.183527 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:05:14.183527 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:05:14.183527 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:05:14.183527 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:05:14.183527 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:05:14.184561 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:05:14.184561 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:05:14.184561 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:05:14.184561 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:05:14.184561 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:05:14.184561 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:05:14.184561 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:05:14.184561 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:05:14.184561 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:05:14.184561 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:05:14.184561 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:05:14.184561 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:05:14.184561 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:05:14.184561 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:05:14.184561 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:05:14.184561 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:05:14.185543 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:05:14.185543 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:05:14.185543 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:05:14.185543 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:05:14.185543 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:05:14.185543 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:05:14.185543 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:05:14.185543 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:05:14.185543 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:05:14.185543 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:05:14.185543 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:05:14.185543 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:05:14.185543 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 607/1195] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 6 16:05:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:05:14.190910 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed3c0aa-dd85-4239-86ce-a14dcc8883e8 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:05:14.220340 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:05:14.303958 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed3c0aa-dd85-4239-86ce-a14dcc8883e8 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:05:14.344883 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bce4d75-6247-4750-8807-8a58cddf8653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:05:14.345494 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce4d75-6247-4750-8807-8a58cddf8653 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:05:14.345763 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce4d75-6247-4750-8807-8a58cddf8653 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:05:14.346031 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce4d75-6247-4750-8807-8a58cddf8653 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:05:14.346699 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bce4d75-6247-4750-8807-8a58cddf8653 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:05:14.346989 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bce4d75-6247-4750-8807-8a58cddf8653 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:05:14.348373 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bce4d75-6247-4750-8807-8a58cddf8653 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:05:14.357443 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 589/1196] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 6 16:05:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:05:14.366530 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324f62b6-a6d7-44fc-865a-63659b86d82a None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:05:14.487352 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324f62b6-a6d7-44fc-865a-63659b86d82a None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:05:14.535519 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453b98f8-5666-4b42-8d91-a25b93b72666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:05:14.536518 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b98f8-5666-4b42-8d91-a25b93b72666 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:05:14.536818 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b98f8-5666-4b42-8d91-a25b93b72666 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:05:14.537097 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b98f8-5666-4b42-8d91-a25b93b72666 None admin] PATH_INFO: `/v3/users/46a4863eb67344a7a479fd3f78ae8927` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:05:14.539580 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453b98f8-5666-4b42-8d91-a25b93b72666 None admin] RBAC: Authorizing `identity:get_user(user_id=46a4863eb67344a7a479fd3f78ae8927)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:05:14.541040 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453b98f8-5666-4b42-8d91-a25b93b72666 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:05:14.544664 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 608/1197] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 6 16:05:14 2022] GET /identity/v3/users/46a4863eb67344a7a479fd3f78ae8927 => generated 311 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:05:14.554439 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bce4d75-6247-4750-8807-8a58cddf8653 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:05:14.659709 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:05:14.681021 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bce4d75-6247-4750-8807-8a58cddf8653 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:05:14.715739 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b17007-1485-44da-a278-76bd9119338d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:05:14.716339 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b17007-1485-44da-a278-76bd9119338d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:05:14.716615 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b17007-1485-44da-a278-76bd9119338d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:05:14.716914 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b17007-1485-44da-a278-76bd9119338d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:05:14.784855 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b17007-1485-44da-a278-76bd9119338d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:05:14.786367 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b17007-1485-44da-a278-76bd9119338d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:05:14.867504 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:05:14.888911 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 590/1198] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 6 16:05:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 16:05:14.895669 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453b98f8-5666-4b42-8d91-a25b93b72666 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:05:14.956036 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:05:14.998297 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453b98f8-5666-4b42-8d91-a25b93b72666 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:05:15.034263 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3249cd66-8b8a-43a9-bc68-9ac0ba1f4c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:05:15.034875 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249cd66-8b8a-43a9-bc68-9ac0ba1f4c5d None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:05:15.035149 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249cd66-8b8a-43a9-bc68-9ac0ba1f4c5d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:05:15.035412 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249cd66-8b8a-43a9-bc68-9ac0ba1f4c5d None admin] PATH_INFO: `/v3/projects/972c629ada64475ba58c29c585d058dc` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:05:15.036689 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3249cd66-8b8a-43a9-bc68-9ac0ba1f4c5d None admin] RBAC: Authorizing `identity:get_project(project_id=972c629ada64475ba58c29c585d058dc)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:05:15.038097 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3249cd66-8b8a-43a9-bc68-9ac0ba1f4c5d None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:05:15.039693 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 609/1199] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 6 16:05:14 2022] GET /identity/v3/projects/972c629ada64475ba58c29c585d058dc => generated 361 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:05:15.046087 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b17007-1485-44da-a278-76bd9119338d None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:05:15.147412 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b17007-1485-44da-a278-76bd9119338d None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:05:15.183777 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1ab22f-0f7d-45cc-942e-1055098c0963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:05:15.184631 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1ab22f-0f7d-45cc-942e-1055098c0963 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:05:15.185061 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1ab22f-0f7d-45cc-942e-1055098c0963 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:05:15.185435 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1ab22f-0f7d-45cc-942e-1055098c0963 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:05:15.186491 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1ab22f-0f7d-45cc-942e-1055098c0963 None admin] RBAC: Adding query filter params (role.id=edaabdea8be84a7980fb2c651fe52f63, scope.project.id=972c629ada64475ba58c29c585d058dc, user.id=46a4863eb67344a7a479fd3f78ae8927) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:05:15.186871 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1ab22f-0f7d-45cc-942e-1055098c0963 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:05:15.189173 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1ab22f-0f7d-45cc-942e-1055098c0963 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:05:15.198837 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 591/1200] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 6 16:05:15 2022] GET /identity/v3/role_assignments?user.id=46a4863eb67344a7a479fd3f78ae8927&scope.project.id=972c629ada64475ba58c29c585d058dc&role.id=edaabdea8be84a7980fb2c651fe52f63 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:05:25.029066 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=37 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:05:25.194165 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=77 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:10.780551 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8597887-da2f-4533-ade4-3b42993e9214 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:10.781174 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8597887-da2f-4533-ade4-3b42993e9214 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:10.781174 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8597887-da2f-4533-ade4-3b42993e9214 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:10.782481 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 610/1201] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:06:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:06:10.788064 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6ab71c-ba63-4d86-b95b-b29513f3fe3e None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:10.788541 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6ab71c-ba63-4d86-b95b-b29513f3fe3e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:10.788632 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6ab71c-ba63-4d86-b95b-b29513f3fe3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:10.810081 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-aa6ab71c-ba63-4d86-b95b-b29513f3fe3e None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:06:10.811277 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-aa6ab71c-ba63-4d86-b95b-b29513f3fe3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:06:10.900788 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 592/1202] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:06:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:06:11.188568 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8597887-da2f-4533-ade4-3b42993e9214 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:11.336075 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8597887-da2f-4533-ade4-3b42993e9214 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:11.375222 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a2f91f-992c-4428-b1b7-65fe8cab52e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:11.375889 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a2f91f-992c-4428-b1b7-65fe8cab52e3 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:11.376163 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a2f91f-992c-4428-b1b7-65fe8cab52e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:11.376427 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a2f91f-992c-4428-b1b7-65fe8cab52e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:11.390539 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a2f91f-992c-4428-b1b7-65fe8cab52e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:11.392271 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a2f91f-992c-4428-b1b7-65fe8cab52e3 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:11.408165 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-34a2f91f-992c-4428-b1b7-65fe8cab52e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:06:11.408165 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:06:11.408165 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:06:11.408165 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:06:11.408165 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:06:11.408165 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:06:11.408165 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:06:11.408165 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:06:11.408165 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:06:11.408165 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:06:11.408165 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:06:11.408165 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:06:11.408165 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:06:11.408165 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:06:11.408165 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:06:11.408165 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:06:11.409354 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:06:11.409354 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:06:11.409354 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:06:11.409354 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:06:11.409354 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:06:11.409354 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:06:11.409354 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:06:11.409354 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:06:11.409354 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:06:11.409354 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:06:11.409354 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:06:11.409354 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:06:11.409354 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:06:11.409354 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:06:11.409354 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:06:11.409354 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:06:11.409354 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:06:11.410740 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:06:11.410740 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:06:11.410740 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:06:11.410740 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 611/1203] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:06:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:06:11.415766 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6ab71c-ba63-4d86-b95b-b29513f3fe3e None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:11.542609 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6ab71c-ba63-4d86-b95b-b29513f3fe3e None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:11.587560 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:11.595712 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891e2b0a-67c1-4d0d-9c31-f9dfba73c6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:11.596340 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e2b0a-67c1-4d0d-9c31-f9dfba73c6b0 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:11.596617 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e2b0a-67c1-4d0d-9c31-f9dfba73c6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:11.596914 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e2b0a-67c1-4d0d-9c31-f9dfba73c6b0 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:11.597570 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891e2b0a-67c1-4d0d-9c31-f9dfba73c6b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:06:11.597866 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891e2b0a-67c1-4d0d-9c31-f9dfba73c6b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:11.599268 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891e2b0a-67c1-4d0d-9c31-f9dfba73c6b0 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:11.611387 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 593/1204] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:06:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:06:11.617574 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a2f91f-992c-4428-b1b7-65fe8cab52e3 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:11.677435 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:11.743771 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a2f91f-992c-4428-b1b7-65fe8cab52e3 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:11.785655 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d46cde-612f-4632-8f7a-8e150520150c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:11.786492 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d46cde-612f-4632-8f7a-8e150520150c None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:11.786823 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d46cde-612f-4632-8f7a-8e150520150c None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:11.787188 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d46cde-612f-4632-8f7a-8e150520150c None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:11.788120 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d46cde-612f-4632-8f7a-8e150520150c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:11.789958 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d46cde-612f-4632-8f7a-8e150520150c None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:11.818824 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 612/1205] 10.222.0.43 () {60 vars in 1132 bytes} [Tue Sep 6 16:06:11 2022] POST /identity/v3/users => generated 251 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:06:13.809334 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025ed902-bbd9-4530-b16b-c8d11434908c None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:13.809927 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025ed902-bbd9-4530-b16b-c8d11434908c None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:13.809927 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025ed902-bbd9-4530-b16b-c8d11434908c None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:13.811321 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 594/1206] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:06:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:06:13.817723 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66170f99-eef1-4c6d-8b34-7cc93a9ecf0b None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:13.818254 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66170f99-eef1-4c6d-8b34-7cc93a9ecf0b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:13.818348 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66170f99-eef1-4c6d-8b34-7cc93a9ecf0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:13.839413 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-66170f99-eef1-4c6d-8b34-7cc93a9ecf0b None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:06:13.840624 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-66170f99-eef1-4c6d-8b34-7cc93a9ecf0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:06:13.913920 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 613/1207] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:06:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:06:14.189720 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025ed902-bbd9-4530-b16b-c8d11434908c None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:14.322662 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025ed902-bbd9-4530-b16b-c8d11434908c None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:14.360163 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3a8476-f00c-4409-807b-b6f9af458752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:14.360831 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3a8476-f00c-4409-807b-b6f9af458752 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:14.361242 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3a8476-f00c-4409-807b-b6f9af458752 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:14.361508 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3a8476-f00c-4409-807b-b6f9af458752 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:14.372734 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3a8476-f00c-4409-807b-b6f9af458752 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:06:14.379796 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-2b3a8476-f00c-4409-807b-b6f9af458752 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:06:14.379796 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:06:14.379796 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:06:14.379796 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:06:14.379796 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:06:14.379796 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:06:14.379796 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:06:14.379796 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:06:14.379796 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:06:14.379796 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:06:14.379796 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:06:14.379796 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:06:14.379796 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:06:14.379796 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:06:14.379796 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:06:14.379796 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:06:14.380810 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:06:14.380810 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:06:14.380810 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:06:14.380810 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:06:14.380810 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:06:14.380810 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:06:14.380810 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:06:14.380810 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:06:14.380810 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:06:14.380810 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:06:14.380810 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:06:14.380810 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:06:14.380810 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:06:14.380810 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:06:14.380810 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:06:14.380810 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:06:14.381699 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:06:14.381699 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:06:14.381699 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:06:14.381699 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:06:14.381699 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:06:14.381699 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:06:14.381699 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:06:14.381699 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:06:14.381699 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:06:14.381699 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:06:14.381699 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:06:14.381699 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:06:14.381699 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 595/1208] 10.222.0.43 () {56 vars in 1102 bytes} [Tue Sep 6 16:06:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:06:14.385163 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66170f99-eef1-4c6d-8b34-7cc93a9ecf0b None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:14.426559 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:14.491421 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66170f99-eef1-4c6d-8b34-7cc93a9ecf0b None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:14.536624 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6db857-64c8-4873-85c3-30c9ed6bd30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:14.537378 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6db857-64c8-4873-85c3-30c9ed6bd30a None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:14.537637 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6db857-64c8-4873-85c3-30c9ed6bd30a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:14.537948 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6db857-64c8-4873-85c3-30c9ed6bd30a None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:14.538658 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6db857-64c8-4873-85c3-30c9ed6bd30a None admin] RBAC: Adding query filter params (name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:06:14.539073 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6db857-64c8-4873-85c3-30c9ed6bd30a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:14.540610 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6db857-64c8-4873-85c3-30c9ed6bd30a None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:14.552272 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 614/1209] 10.222.0.43 () {56 vars in 1103 bytes} [Tue Sep 6 16:06:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:06:14.559057 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3a8476-f00c-4409-807b-b6f9af458752 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:14.623981 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:14.694774 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3a8476-f00c-4409-807b-b6f9af458752 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:14.737439 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f3e17e-8bf9-4296-a011-19ead36cc0f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:14.738147 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f3e17e-8bf9-4296-a011-19ead36cc0f8 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:14.738450 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f3e17e-8bf9-4296-a011-19ead36cc0f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:14.738931 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f3e17e-8bf9-4296-a011-19ead36cc0f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:14.753084 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f3e17e-8bf9-4296-a011-19ead36cc0f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:14.754530 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f3e17e-8bf9-4296-a011-19ead36cc0f8 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:14.773818 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-a7f3e17e-8bf9-4296-a011-19ead36cc0f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:06:14.773818 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:06:14.773818 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:06:14.773818 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:06:14.773818 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:06:14.773818 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:06:14.773818 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:06:14.773818 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:06:14.773818 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:06:14.773818 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:06:14.773818 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:06:14.773818 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:06:14.773818 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:06:14.773818 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:06:14.773818 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:06:14.773818 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:06:14.776808 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:06:14.776808 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:06:14.776808 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:06:14.776808 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:06:14.776808 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:06:14.776808 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:06:14.776808 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:06:14.776808 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:06:14.776808 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:06:14.776808 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:06:14.776808 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:06:14.776808 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:06:14.776808 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:06:14.776808 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:06:14.776808 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:06:14.776808 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:06:14.776808 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:06:14.777963 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:06:14.777963 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:06:14.777963 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:06:14.777963 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 596/1210] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:06:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:06:14.781725 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6db857-64c8-4873-85c3-30c9ed6bd30a None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:14.906928 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:14.912377 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6db857-64c8-4873-85c3-30c9ed6bd30a None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:14.962735 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93042347-9428-4ed4-8ddd-12bdb7ecf252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:14.963442 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93042347-9428-4ed4-8ddd-12bdb7ecf252 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:14.963784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93042347-9428-4ed4-8ddd-12bdb7ecf252 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:14.964288 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93042347-9428-4ed4-8ddd-12bdb7ecf252 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:14.965203 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93042347-9428-4ed4-8ddd-12bdb7ecf252 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:06:14.965510 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93042347-9428-4ed4-8ddd-12bdb7ecf252 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:14.967219 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93042347-9428-4ed4-8ddd-12bdb7ecf252 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:14.982040 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 615/1211] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:06:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:06:14.988019 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f3e17e-8bf9-4296-a011-19ead36cc0f8 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:15.109168 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f3e17e-8bf9-4296-a011-19ead36cc0f8 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:15.110947 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:15.145768 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3748bd-4db7-4853-b5a3-87a51e96af6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:15.146404 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3748bd-4db7-4853-b5a3-87a51e96af6e None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:15.146688 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3748bd-4db7-4853-b5a3-87a51e96af6e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:15.146981 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3748bd-4db7-4853-b5a3-87a51e96af6e None admin] PATH_INFO: `/v3/users/placement` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:15.155754 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3748bd-4db7-4853-b5a3-87a51e96af6e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:15.157338 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3748bd-4db7-4853-b5a3-87a51e96af6e None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:15.169572 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-0d3748bd-4db7-4853-b5a3-87a51e96af6e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 06 16:06:15.169572 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:06:15.169572 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:06:15.169572 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:06:15.169572 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:06:15.169572 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:06:15.169572 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:06:15.169572 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:06:15.169572 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:06:15.169572 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:06:15.169572 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:06:15.169572 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:06:15.169572 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:06:15.169572 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:06:15.169572 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:06:15.169572 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:06:15.170721 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:06:15.170721 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:06:15.170721 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:06:15.170721 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:06:15.170721 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:06:15.170721 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:06:15.170721 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:06:15.170721 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:06:15.170721 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:06:15.170721 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:06:15.170721 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:06:15.170721 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:06:15.170721 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:06:15.170721 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:06:15.170721 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:06:15.170721 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:06:15.170721 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:06:15.170721 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:06:15.177294 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:06:15.177294 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:06:15.177294 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:06:15.177294 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:06:15.177294 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:06:15.177294 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:06:15.177294 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:06:15.177294 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:06:15.177294 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 06 16:06:15.177294 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:06:15.177294 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 597/1212] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:06:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:06:15.177936 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93042347-9428-4ed4-8ddd-12bdb7ecf252 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:15.304623 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93042347-9428-4ed4-8ddd-12bdb7ecf252 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:15.350097 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604ce890-90a5-4990-b0aa-65815f9a2cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:15.350724 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ce890-90a5-4990-b0aa-65815f9a2cf2 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:15.350997 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ce890-90a5-4990-b0aa-65815f9a2cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:15.351263 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ce890-90a5-4990-b0aa-65815f9a2cf2 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:15.351984 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604ce890-90a5-4990-b0aa-65815f9a2cf2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:06:15.352280 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604ce890-90a5-4990-b0aa-65815f9a2cf2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:15.353751 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604ce890-90a5-4990-b0aa-65815f9a2cf2 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:15.381942 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 616/1213] 10.222.0.43 () {56 vars in 1143 bytes} [Tue Sep 6 16:06:15 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:06:15.382842 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3748bd-4db7-4853-b5a3-87a51e96af6e None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:15.502118 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3748bd-4db7-4853-b5a3-87a51e96af6e None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:15.553494 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c61125-81e2-4009-a89c-3c7f6062044b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:15.554094 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c61125-81e2-4009-a89c-3c7f6062044b None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:15.554443 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c61125-81e2-4009-a89c-3c7f6062044b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:15.554720 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c61125-81e2-4009-a89c-3c7f6062044b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:15.593145 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:15.637210 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c61125-81e2-4009-a89c-3c7f6062044b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:15.638663 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c61125-81e2-4009-a89c-3c7f6062044b None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:15.761880 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 598/1214] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 6 16:06:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 16:06:15.768040 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604ce890-90a5-4990-b0aa-65815f9a2cf2 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:15.790013 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:15.891404 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604ce890-90a5-4990-b0aa-65815f9a2cf2 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:15.933919 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d57ab42-1de2-4c7e-9a0c-8a07c2fb3230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:15.934941 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d57ab42-1de2-4c7e-9a0c-8a07c2fb3230 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:15.935323 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d57ab42-1de2-4c7e-9a0c-8a07c2fb3230 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:15.935697 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d57ab42-1de2-4c7e-9a0c-8a07c2fb3230 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:15.949086 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d57ab42-1de2-4c7e-9a0c-8a07c2fb3230 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:15.951550 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d57ab42-1de2-4c7e-9a0c-8a07c2fb3230 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:15.971953 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-3d57ab42-1de2-4c7e-9a0c-8a07c2fb3230 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:06:15.971953 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:06:15.971953 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:06:15.971953 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:06:15.971953 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:06:15.971953 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:06:15.971953 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:06:15.971953 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:06:15.971953 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:06:15.971953 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:06:15.971953 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:06:15.971953 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:06:15.971953 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:06:15.971953 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:06:15.971953 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:06:15.971953 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:06:15.973269 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:06:15.973269 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:06:15.973269 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:06:15.973269 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:06:15.973269 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:06:15.973269 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:06:15.973269 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:06:15.973269 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:06:15.973269 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:06:15.973269 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:06:15.973269 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:06:15.973269 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:06:15.973269 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:06:15.973269 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:06:15.973269 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:06:15.973269 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:06:15.973269 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:06:15.974282 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:06:15.974282 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:06:15.974282 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:06:15.974282 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 617/1215] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:06:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:06:15.980803 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c61125-81e2-4009-a89c-3c7f6062044b None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:16.051335 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:16.097982 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c61125-81e2-4009-a89c-3c7f6062044b None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:16.134894 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f19e6b-3fba-467c-a4e8-6a6ad0f1b4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:16.135518 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f19e6b-3fba-467c-a4e8-6a6ad0f1b4d1 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:16.135795 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f19e6b-3fba-467c-a4e8-6a6ad0f1b4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:16.136069 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f19e6b-3fba-467c-a4e8-6a6ad0f1b4d1 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:16.136720 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f19e6b-3fba-467c-a4e8-6a6ad0f1b4d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:06:16.137054 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f19e6b-3fba-467c-a4e8-6a6ad0f1b4d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:16.138532 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f19e6b-3fba-467c-a4e8-6a6ad0f1b4d1 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:16.156919 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 599/1216] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:06:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:06:16.158468 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d57ab42-1de2-4c7e-9a0c-8a07c2fb3230 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:16.250263 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:16.290797 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d57ab42-1de2-4c7e-9a0c-8a07c2fb3230 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:16.334674 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe6fdfe-c84e-4d0c-8c82-1210fee6601e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:16.335450 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe6fdfe-c84e-4d0c-8c82-1210fee6601e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:16.335726 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe6fdfe-c84e-4d0c-8c82-1210fee6601e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:16.336002 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe6fdfe-c84e-4d0c-8c82-1210fee6601e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:16.343510 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe6fdfe-c84e-4d0c-8c82-1210fee6601e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:16.345183 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe6fdfe-c84e-4d0c-8c82-1210fee6601e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:16.356885 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-ebe6fdfe-c84e-4d0c-8c82-1210fee6601e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:06:16.356885 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:06:16.356885 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:06:16.356885 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:06:16.356885 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:06:16.356885 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:06:16.356885 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:06:16.356885 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:06:16.356885 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:06:16.356885 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:06:16.356885 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:06:16.356885 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:06:16.356885 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:06:16.356885 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:06:16.356885 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:06:16.357862 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:06:16.357862 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:06:16.357862 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:06:16.357862 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:06:16.357862 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:06:16.357862 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:06:16.357862 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:06:16.357862 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:06:16.357862 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:06:16.357862 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:06:16.357862 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:06:16.357862 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:06:16.357862 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:06:16.357862 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:06:16.357862 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:06:16.357862 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:06:16.357862 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:06:16.358811 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:06:16.358811 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:06:16.358811 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:06:16.358811 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:06:16.358811 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:06:16.358811 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:06:16.358811 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:06:16.358811 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:06:16.358811 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 618/1217] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 6 16:06:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:06:16.364076 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f19e6b-3fba-467c-a4e8-6a6ad0f1b4d1 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:16.472244 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f19e6b-3fba-467c-a4e8-6a6ad0f1b4d1 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:16.498502 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:16.517458 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59819e55-349e-40ca-9989-66877cc93296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:16.518460 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59819e55-349e-40ca-9989-66877cc93296 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:16.518886 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59819e55-349e-40ca-9989-66877cc93296 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:16.519300 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59819e55-349e-40ca-9989-66877cc93296 None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:16.520422 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59819e55-349e-40ca-9989-66877cc93296 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:06:16.520804 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59819e55-349e-40ca-9989-66877cc93296 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:16.522637 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59819e55-349e-40ca-9989-66877cc93296 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:16.541408 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 600/1218] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 6 16:06:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:06:16.543243 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe6fdfe-c84e-4d0c-8c82-1210fee6601e None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:16.653280 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe6fdfe-c84e-4d0c-8c82-1210fee6601e None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:16.686122 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:16.693406 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3375282f-d6a8-4919-a3c8-6c3aea5b79e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:16.694010 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3375282f-d6a8-4919-a3c8-6c3aea5b79e7 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:16.694457 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3375282f-d6a8-4919-a3c8-6c3aea5b79e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:16.694738 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3375282f-d6a8-4919-a3c8-6c3aea5b79e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:16.695459 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3375282f-d6a8-4919-a3c8-6c3aea5b79e7 None admin] RBAC: Adding query filter params (role.id=31321627b02c4ff188f8fe1521ce3181, scope.project.id=128d210e8dad48ac921ed4cb1d444ceb, user.id=71c2912a44f04bd983b4d4112dbee734) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:06:16.695835 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3375282f-d6a8-4919-a3c8-6c3aea5b79e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:16.697444 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3375282f-d6a8-4919-a3c8-6c3aea5b79e7 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:16.716801 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 619/1219] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 6 16:06:16 2022] GET /identity/v3/role_assignments?user.id=71c2912a44f04bd983b4d4112dbee734&scope.project.id=128d210e8dad48ac921ed4cb1d444ceb&role.id=31321627b02c4ff188f8fe1521ce3181 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:06:18.690719 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9691a42e-6809-4354-9821-875e5ceb934a None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:18.691313 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9691a42e-6809-4354-9821-875e5ceb934a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:18.691313 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9691a42e-6809-4354-9821-875e5ceb934a None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:18.692505 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 601/1220] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:06:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:06:18.697988 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dbd802-b2ce-49ff-a821-3a0c4fcc0ff0 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:18.698477 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dbd802-b2ce-49ff-a821-3a0c4fcc0ff0 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:18.698557 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dbd802-b2ce-49ff-a821-3a0c4fcc0ff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:18.719236 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-c5dbd802-b2ce-49ff-a821-3a0c4fcc0ff0 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:06:18.720669 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c5dbd802-b2ce-49ff-a821-3a0c4fcc0ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:06:18.794915 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 620/1221] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:06:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:06:19.069632 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9691a42e-6809-4354-9821-875e5ceb934a None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:19.184531 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9691a42e-6809-4354-9821-875e5ceb934a None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:19.223383 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6554f9ce-a41d-4093-9dff-109cab1d499f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:19.224048 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6554f9ce-a41d-4093-9dff-109cab1d499f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:19.224352 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6554f9ce-a41d-4093-9dff-109cab1d499f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:19.224612 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6554f9ce-a41d-4093-9dff-109cab1d499f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:19.237657 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6554f9ce-a41d-4093-9dff-109cab1d499f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:06:19.243126 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-6554f9ce-a41d-4093-9dff-109cab1d499f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:06:19.243126 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:06:19.243126 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:06:19.243126 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:06:19.243126 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:06:19.243126 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:06:19.243126 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:06:19.243126 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:06:19.243126 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:06:19.243126 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:06:19.243126 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:06:19.243126 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:06:19.243126 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:06:19.243126 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:06:19.243126 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:06:19.243126 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:06:19.244167 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:06:19.244167 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:06:19.244167 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:06:19.244167 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:06:19.244167 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:06:19.244167 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:06:19.244167 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:06:19.244167 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:06:19.244167 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:06:19.244167 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:06:19.244167 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:06:19.244167 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:06:19.244167 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:06:19.244167 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:06:19.244167 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:06:19.244167 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:06:19.249841 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:06:19.249841 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:06:19.249841 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:06:19.249841 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:06:19.249841 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:06:19.249841 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:06:19.249841 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:06:19.249841 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:06:19.249841 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:06:19.249841 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:06:19.249841 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:06:19.249841 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:06:19.249841 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 602/1222] 10.222.0.43 () {56 vars in 1102 bytes} [Tue Sep 6 16:06:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:06:19.251545 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dbd802-b2ce-49ff-a821-3a0c4fcc0ff0 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:19.356567 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dbd802-b2ce-49ff-a821-3a0c4fcc0ff0 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:19.379556 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:19.390635 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df31a449-a9ae-42d6-aa10-7f079a7770ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:19.391250 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df31a449-a9ae-42d6-aa10-7f079a7770ec None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:19.391527 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df31a449-a9ae-42d6-aa10-7f079a7770ec None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:19.391796 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df31a449-a9ae-42d6-aa10-7f079a7770ec None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:19.392462 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df31a449-a9ae-42d6-aa10-7f079a7770ec None admin] RBAC: Adding query filter params (name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:06:19.392802 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df31a449-a9ae-42d6-aa10-7f079a7770ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:19.394197 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df31a449-a9ae-42d6-aa10-7f079a7770ec None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:19.402382 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 621/1223] 10.222.0.43 () {56 vars in 1103 bytes} [Tue Sep 6 16:06:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:06:19.408261 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6554f9ce-a41d-4093-9dff-109cab1d499f None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:19.457018 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:19.514772 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6554f9ce-a41d-4093-9dff-109cab1d499f None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:19.551056 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf05ade0-fdac-46fb-9f06-bb3b26f4de33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:19.551692 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf05ade0-fdac-46fb-9f06-bb3b26f4de33 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:19.551976 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf05ade0-fdac-46fb-9f06-bb3b26f4de33 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:19.552243 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf05ade0-fdac-46fb-9f06-bb3b26f4de33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:19.562668 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf05ade0-fdac-46fb-9f06-bb3b26f4de33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:19.564154 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf05ade0-fdac-46fb-9f06-bb3b26f4de33 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:19.580135 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-bf05ade0-fdac-46fb-9f06-bb3b26f4de33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:06:19.580135 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:06:19.580135 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:06:19.580135 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:06:19.580135 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:06:19.580135 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:06:19.580135 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:06:19.580135 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:06:19.580135 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:06:19.580135 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:06:19.580135 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:06:19.580135 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:06:19.580135 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:06:19.580135 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:06:19.580135 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:06:19.580135 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:06:19.581403 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:06:19.581403 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:06:19.581403 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:06:19.581403 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:06:19.581403 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:06:19.581403 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:06:19.581403 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:06:19.581403 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:06:19.581403 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:06:19.581403 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:06:19.581403 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:06:19.581403 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:06:19.581403 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:06:19.581403 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:06:19.581403 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:06:19.581403 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:06:19.581403 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:06:19.582513 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:06:19.582513 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:06:19.582513 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:06:19.582513 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 603/1224] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:06:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:06:19.587329 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df31a449-a9ae-42d6-aa10-7f079a7770ec None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:19.691982 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df31a449-a9ae-42d6-aa10-7f079a7770ec None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:19.735008 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4249900-c74f-4ba7-9463-4d975eff7fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:19.735684 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4249900-c74f-4ba7-9463-4d975eff7fd5 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:19.735967 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4249900-c74f-4ba7-9463-4d975eff7fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:19.736269 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4249900-c74f-4ba7-9463-4d975eff7fd5 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:19.736980 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4249900-c74f-4ba7-9463-4d975eff7fd5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:06:19.737319 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4249900-c74f-4ba7-9463-4d975eff7fd5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:19.738786 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4249900-c74f-4ba7-9463-4d975eff7fd5 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:19.750979 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 622/1225] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:06:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:06:19.756975 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf05ade0-fdac-46fb-9f06-bb3b26f4de33 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:19.857036 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:19.863980 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf05ade0-fdac-46fb-9f06-bb3b26f4de33 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:19.900365 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa48ac02-a8c0-4f89-8fb4-af7b3f9e6a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:19.901028 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa48ac02-a8c0-4f89-8fb4-af7b3f9e6a4a None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:19.901318 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa48ac02-a8c0-4f89-8fb4-af7b3f9e6a4a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:19.901607 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa48ac02-a8c0-4f89-8fb4-af7b3f9e6a4a None admin] PATH_INFO: `/v3/users/placement` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:19.909300 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa48ac02-a8c0-4f89-8fb4-af7b3f9e6a4a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:19.910809 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa48ac02-a8c0-4f89-8fb4-af7b3f9e6a4a None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:19.922273 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-aa48ac02-a8c0-4f89-8fb4-af7b3f9e6a4a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 06 16:06:19.922273 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:06:19.922273 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:06:19.922273 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:06:19.922273 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:06:19.922273 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:06:19.922273 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:06:19.922273 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:06:19.922273 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:06:19.922273 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:06:19.922273 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:06:19.922273 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:06:19.922273 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:06:19.922273 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:06:19.922273 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:06:19.922273 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:06:19.923385 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:06:19.923385 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:06:19.923385 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:06:19.923385 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:06:19.923385 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:06:19.923385 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:06:19.923385 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:06:19.923385 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:06:19.923385 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:06:19.923385 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:06:19.923385 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:06:19.923385 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:06:19.923385 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:06:19.923385 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:06:19.923385 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:06:19.923385 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:06:19.923385 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:06:19.923385 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:06:19.924328 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:06:19.924328 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:06:19.924328 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:06:19.924328 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:06:19.924328 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:06:19.924328 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:06:19.924328 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:06:19.924328 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:06:19.924328 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 06 16:06:19.924328 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:06:19.928614 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 604/1226] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:06:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:06:19.929878 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4249900-c74f-4ba7-9463-4d975eff7fd5 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:19.982807 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:20.043897 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4249900-c74f-4ba7-9463-4d975eff7fd5 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:20.081541 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770b38f2-77ad-469d-b81c-d7a9e01ecaf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:20.082240 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770b38f2-77ad-469d-b81c-d7a9e01ecaf6 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:20.082538 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770b38f2-77ad-469d-b81c-d7a9e01ecaf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:20.082839 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770b38f2-77ad-469d-b81c-d7a9e01ecaf6 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:20.083610 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770b38f2-77ad-469d-b81c-d7a9e01ecaf6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:06:20.083981 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770b38f2-77ad-469d-b81c-d7a9e01ecaf6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:20.085654 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770b38f2-77ad-469d-b81c-d7a9e01ecaf6 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:20.104020 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 623/1227] 10.222.0.43 () {56 vars in 1143 bytes} [Tue Sep 6 16:06:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:06:20.109941 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa48ac02-a8c0-4f89-8fb4-af7b3f9e6a4a None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:20.223726 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa48ac02-a8c0-4f89-8fb4-af7b3f9e6a4a None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:20.262420 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca965ae-6e00-44cb-8d34-749ecb88747c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:20.263005 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca965ae-6e00-44cb-8d34-749ecb88747c None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:20.263332 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca965ae-6e00-44cb-8d34-749ecb88747c None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:20.263643 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca965ae-6e00-44cb-8d34-749ecb88747c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:20.278921 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca965ae-6e00-44cb-8d34-749ecb88747c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:20.280407 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca965ae-6e00-44cb-8d34-749ecb88747c None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:20.296486 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-3ca965ae-6e00-44cb-8d34-749ecb88747c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:06:20.296486 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:06:20.296486 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:06:20.296486 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:06:20.296486 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:06:20.296486 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:06:20.296486 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:06:20.296486 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:06:20.296486 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:06:20.296486 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:06:20.296486 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:06:20.296486 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:06:20.296486 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:06:20.296486 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:06:20.296486 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:06:20.296486 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:06:20.297575 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:06:20.297575 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:06:20.297575 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:06:20.297575 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:06:20.297575 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:06:20.297575 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:06:20.297575 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:06:20.297575 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:06:20.297575 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:06:20.297575 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:06:20.297575 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:06:20.297575 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:06:20.297575 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:06:20.297575 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:06:20.297575 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:06:20.297575 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:06:20.297575 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:06:20.298484 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:06:20.298484 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:06:20.298484 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:06:20.298484 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 605/1228] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:06:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:06:20.303827 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770b38f2-77ad-469d-b81c-d7a9e01ecaf6 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:20.413025 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770b38f2-77ad-469d-b81c-d7a9e01ecaf6 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:20.415756 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:20.451053 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc6b287-73b0-45e1-a613-dd47503a5a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:20.451801 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc6b287-73b0-45e1-a613-dd47503a5a9c None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:20.452082 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc6b287-73b0-45e1-a613-dd47503a5a9c None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:20.452417 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc6b287-73b0-45e1-a613-dd47503a5a9c None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:20.453107 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc6b287-73b0-45e1-a613-dd47503a5a9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:06:20.453406 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc6b287-73b0-45e1-a613-dd47503a5a9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:20.454941 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc6b287-73b0-45e1-a613-dd47503a5a9c None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:20.466760 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 624/1229] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:06:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:06:20.472595 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca965ae-6e00-44cb-8d34-749ecb88747c None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:20.482879 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:20.585962 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca965ae-6e00-44cb-8d34-749ecb88747c None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:20.627644 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ed3a85-cf5a-459d-98f1-d5071139ef7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:20.628282 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ed3a85-cf5a-459d-98f1-d5071139ef7e None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:20.628563 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ed3a85-cf5a-459d-98f1-d5071139ef7e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:20.628869 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ed3a85-cf5a-459d-98f1-d5071139ef7e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:20.636024 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ed3a85-cf5a-459d-98f1-d5071139ef7e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:20.637670 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ed3a85-cf5a-459d-98f1-d5071139ef7e None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:20.649903 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-57ed3a85-cf5a-459d-98f1-d5071139ef7e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:06:20.649903 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:06:20.649903 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:06:20.649903 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:06:20.649903 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:06:20.649903 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:06:20.649903 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:06:20.649903 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:06:20.649903 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:06:20.649903 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:06:20.649903 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:06:20.649903 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:06:20.649903 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:06:20.649903 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:06:20.649903 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:06:20.651639 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:06:20.651639 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:06:20.651639 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:06:20.651639 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:06:20.651639 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:06:20.651639 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:06:20.651639 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:06:20.651639 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:06:20.651639 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:06:20.651639 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:06:20.651639 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:06:20.651639 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:06:20.651639 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:06:20.651639 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:06:20.651639 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:06:20.651639 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:06:20.651639 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:06:20.653404 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:06:20.653404 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:06:20.653404 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:06:20.653404 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:06:20.653404 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:06:20.653404 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:06:20.653404 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:06:20.653404 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:06:20.653404 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 606/1230] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 6 16:06:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:06:20.657317 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc6b287-73b0-45e1-a613-dd47503a5a9c None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:20.777563 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc6b287-73b0-45e1-a613-dd47503a5a9c None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:20.814491 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4ad003-23c1-4f50-9f1a-012e18b4592d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:20.815095 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ad003-23c1-4f50-9f1a-012e18b4592d None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:20.815365 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ad003-23c1-4f50-9f1a-012e18b4592d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:20.815625 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ad003-23c1-4f50-9f1a-012e18b4592d None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:20.816305 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4ad003-23c1-4f50-9f1a-012e18b4592d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:06:20.816602 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4ad003-23c1-4f50-9f1a-012e18b4592d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:20.818081 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4ad003-23c1-4f50-9f1a-012e18b4592d None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:20.831186 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 625/1231] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 6 16:06:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:06:20.838539 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ed3a85-cf5a-459d-98f1-d5071139ef7e None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:20.908202 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:20.944132 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ed3a85-cf5a-459d-98f1-d5071139ef7e None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:20.979450 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966cdb21-480f-4e3f-b5f5-aba9e6fb0b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:20.980102 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966cdb21-480f-4e3f-b5f5-aba9e6fb0b3b None admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:20.980373 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966cdb21-480f-4e3f-b5f5-aba9e6fb0b3b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:20.980643 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966cdb21-480f-4e3f-b5f5-aba9e6fb0b3b None admin] PATH_INFO: `/v3/projects/128d210e8dad48ac921ed4cb1d444ceb/users/71c2912a44f04bd983b4d4112dbee734/roles/31321627b02c4ff188f8fe1521ce3181` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:20.996693 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966cdb21-480f-4e3f-b5f5-aba9e6fb0b3b None admin] RBAC: Authorizing `identity:create_grant(project_id=128d210e8dad48ac921ed4cb1d444ceb, user_id=71c2912a44f04bd983b4d4112dbee734, role_id=31321627b02c4ff188f8fe1521ce3181)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:20.998302 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966cdb21-480f-4e3f-b5f5-aba9e6fb0b3b None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:21.011100 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 607/1232] 10.222.0.43 () {58 vars in 1439 bytes} [Tue Sep 6 16:06:20 2022] PUT /identity/v3/projects/128d210e8dad48ac921ed4cb1d444ceb/users/71c2912a44f04bd983b4d4112dbee734/roles/31321627b02c4ff188f8fe1521ce3181 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:06:22.979009 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2a6505-681e-4d2c-bae2-8c3faf096bfc None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:22.979744 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2a6505-681e-4d2c-bae2-8c3faf096bfc None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:22.979870 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2a6505-681e-4d2c-bae2-8c3faf096bfc None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:22.981384 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 626/1233] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:06:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:06:22.990832 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5d87de-643a-486c-9c8a-cf8389d2736a None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:22.991242 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5d87de-643a-486c-9c8a-cf8389d2736a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:22.991384 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5d87de-643a-486c-9c8a-cf8389d2736a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:23.012094 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-6a5d87de-643a-486c-9c8a-cf8389d2736a None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:06:23.013336 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6a5d87de-643a-486c-9c8a-cf8389d2736a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:06:23.085806 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 608/1234] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:06:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:06:23.387084 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2a6505-681e-4d2c-bae2-8c3faf096bfc None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:23.415021 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:23.494403 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2a6505-681e-4d2c-bae2-8c3faf096bfc None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:23.529483 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9b88e0-f504-4689-9d10-a8233de8fe56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:23.530042 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9b88e0-f504-4689-9d10-a8233de8fe56 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:23.530315 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9b88e0-f504-4689-9d10-a8233de8fe56 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:23.530578 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9b88e0-f504-4689-9d10-a8233de8fe56 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:23.540803 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9b88e0-f504-4689-9d10-a8233de8fe56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:06:23.545887 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-0b9b88e0-f504-4689-9d10-a8233de8fe56 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:06:23.545887 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:06:23.545887 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:06:23.545887 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:06:23.545887 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:06:23.545887 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:06:23.545887 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:06:23.545887 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:06:23.545887 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:06:23.545887 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:06:23.545887 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:06:23.545887 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:06:23.545887 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:06:23.545887 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:06:23.545887 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:06:23.545887 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:06:23.547318 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:06:23.547318 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:06:23.547318 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:06:23.547318 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:06:23.547318 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:06:23.547318 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:06:23.547318 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:06:23.547318 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:06:23.547318 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:06:23.547318 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:06:23.547318 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:06:23.547318 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:06:23.547318 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:06:23.547318 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:06:23.547318 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:06:23.547318 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:06:23.548467 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:06:23.548467 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:06:23.548467 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:06:23.548467 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:06:23.548467 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:06:23.548467 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:06:23.548467 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:06:23.548467 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:06:23.548467 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:06:23.548467 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:06:23.548467 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:06:23.548467 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:06:23.548467 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 627/1235] 10.222.0.43 () {56 vars in 1102 bytes} [Tue Sep 6 16:06:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:06:23.553347 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5d87de-643a-486c-9c8a-cf8389d2736a None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:23.614277 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:23.674040 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5d87de-643a-486c-9c8a-cf8389d2736a None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:23.719508 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac47d17-2b0b-483e-8a4f-a6773f2d1b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:23.720531 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac47d17-2b0b-483e-8a4f-a6773f2d1b69 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:23.720981 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac47d17-2b0b-483e-8a4f-a6773f2d1b69 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:23.721333 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac47d17-2b0b-483e-8a4f-a6773f2d1b69 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:23.722785 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac47d17-2b0b-483e-8a4f-a6773f2d1b69 None admin] RBAC: Adding query filter params (name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:06:23.723534 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac47d17-2b0b-483e-8a4f-a6773f2d1b69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:23.725874 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac47d17-2b0b-483e-8a4f-a6773f2d1b69 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:23.738670 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 609/1236] 10.222.0.43 () {56 vars in 1103 bytes} [Tue Sep 6 16:06:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:06:23.744886 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9b88e0-f504-4689-9d10-a8233de8fe56 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:23.838576 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:23.855754 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9b88e0-f504-4689-9d10-a8233de8fe56 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:23.893375 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a205464-3f7d-42a8-a15c-4b57cd322cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:23.894011 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a205464-3f7d-42a8-a15c-4b57cd322cba None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:23.894288 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a205464-3f7d-42a8-a15c-4b57cd322cba None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:23.894557 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a205464-3f7d-42a8-a15c-4b57cd322cba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:23.905263 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a205464-3f7d-42a8-a15c-4b57cd322cba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:23.906803 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a205464-3f7d-42a8-a15c-4b57cd322cba None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:23.921689 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-4a205464-3f7d-42a8-a15c-4b57cd322cba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:06:23.921689 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:06:23.921689 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:06:23.921689 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:06:23.921689 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:06:23.921689 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:06:23.921689 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:06:23.921689 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:06:23.921689 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:06:23.921689 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:06:23.921689 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:06:23.921689 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:06:23.921689 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:06:23.921689 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:06:23.921689 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:06:23.921689 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:06:23.922819 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:06:23.922819 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:06:23.922819 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:06:23.922819 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:06:23.922819 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:06:23.922819 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:06:23.922819 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:06:23.922819 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:06:23.922819 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:06:23.922819 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:06:23.922819 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:06:23.922819 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:06:23.922819 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:06:23.922819 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:06:23.922819 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:06:23.922819 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:06:23.922819 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:06:23.924240 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:06:23.924240 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:06:23.924240 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:06:23.924240 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 628/1237] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:06:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:06:23.928874 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac47d17-2b0b-483e-8a4f-a6773f2d1b69 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:24.041240 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac47d17-2b0b-483e-8a4f-a6773f2d1b69 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:24.055370 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:24.088002 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf79150b-3fc6-4cd5-aca9-a5f0093faa9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:24.088807 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf79150b-3fc6-4cd5-aca9-a5f0093faa9c None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:24.089031 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf79150b-3fc6-4cd5-aca9-a5f0093faa9c None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:24.089322 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf79150b-3fc6-4cd5-aca9-a5f0093faa9c None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:24.089973 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf79150b-3fc6-4cd5-aca9-a5f0093faa9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:06:24.090270 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf79150b-3fc6-4cd5-aca9-a5f0093faa9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:24.091679 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf79150b-3fc6-4cd5-aca9-a5f0093faa9c None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:24.104111 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 610/1238] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:06:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:06:24.110094 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a205464-3f7d-42a8-a15c-4b57cd322cba None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:24.218810 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a205464-3f7d-42a8-a15c-4b57cd322cba None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:24.256363 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3c5350-442e-4757-961b-0d540118f793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:24.257046 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3c5350-442e-4757-961b-0d540118f793 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:24.257325 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3c5350-442e-4757-961b-0d540118f793 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:24.257593 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3c5350-442e-4757-961b-0d540118f793 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:24.260790 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:24.267519 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3c5350-442e-4757-961b-0d540118f793 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:24.269643 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3c5350-442e-4757-961b-0d540118f793 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:24.283429 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-ff3c5350-442e-4757-961b-0d540118f793 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 06 16:06:24.283429 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:06:24.283429 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:06:24.283429 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:06:24.283429 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:06:24.283429 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:06:24.283429 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:06:24.283429 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:06:24.283429 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:06:24.283429 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:06:24.283429 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:06:24.283429 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:06:24.283429 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:06:24.283429 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:06:24.283429 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:06:24.283429 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:06:24.284602 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:06:24.284602 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:06:24.284602 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:06:24.284602 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:06:24.284602 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:06:24.284602 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:06:24.284602 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:06:24.284602 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:06:24.284602 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:06:24.284602 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:06:24.284602 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:06:24.284602 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:06:24.284602 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:06:24.284602 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:06:24.284602 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:06:24.284602 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:06:24.284602 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:06:24.284602 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:06:24.286163 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:06:24.286163 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:06:24.286163 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:06:24.286163 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:06:24.286163 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:06:24.286163 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:06:24.286163 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:06:24.286163 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:06:24.286163 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 06 16:06:24.286163 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:06:24.286163 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 629/1239] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:06:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:06:24.290867 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf79150b-3fc6-4cd5-aca9-a5f0093faa9c None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:24.396941 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf79150b-3fc6-4cd5-aca9-a5f0093faa9c None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:24.431595 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced7ae9-65a2-4887-a1bd-f91ded11ad50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:24.432295 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced7ae9-65a2-4887-a1bd-f91ded11ad50 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:24.432499 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced7ae9-65a2-4887-a1bd-f91ded11ad50 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:24.432746 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced7ae9-65a2-4887-a1bd-f91ded11ad50 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:24.433518 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ced7ae9-65a2-4887-a1bd-f91ded11ad50 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:06:24.433810 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ced7ae9-65a2-4887-a1bd-f91ded11ad50 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:24.435317 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ced7ae9-65a2-4887-a1bd-f91ded11ad50 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:24.453240 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 611/1240] 10.222.0.43 () {56 vars in 1143 bytes} [Tue Sep 6 16:06:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:06:24.459187 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3c5350-442e-4757-961b-0d540118f793 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:24.568010 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3c5350-442e-4757-961b-0d540118f793 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:24.608112 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fe0ba8-8b29-487d-97d0-04a34845cd28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:24.608713 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe0ba8-8b29-487d-97d0-04a34845cd28 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:24.609018 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe0ba8-8b29-487d-97d0-04a34845cd28 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:24.609286 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe0ba8-8b29-487d-97d0-04a34845cd28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:24.681042 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:24.689082 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fe0ba8-8b29-487d-97d0-04a34845cd28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:24.690660 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fe0ba8-8b29-487d-97d0-04a34845cd28 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:24.810368 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 630/1241] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 6 16:06:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 16:06:24.816391 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced7ae9-65a2-4887-a1bd-f91ded11ad50 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:24.846850 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:24.923591 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced7ae9-65a2-4887-a1bd-f91ded11ad50 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:24.965581 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3753944d-b826-450b-86b0-6a17b5777f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:24.966204 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3753944d-b826-450b-86b0-6a17b5777f92 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:24.966482 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3753944d-b826-450b-86b0-6a17b5777f92 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:24.966801 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3753944d-b826-450b-86b0-6a17b5777f92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:24.978905 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3753944d-b826-450b-86b0-6a17b5777f92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:24.980461 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3753944d-b826-450b-86b0-6a17b5777f92 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:24.995536 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-3753944d-b826-450b-86b0-6a17b5777f92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:06:24.995536 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:06:24.995536 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:06:24.995536 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:06:24.995536 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:06:24.995536 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:06:24.995536 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:06:24.995536 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:06:24.995536 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:06:24.995536 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:06:24.995536 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:06:24.995536 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:06:24.995536 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:06:24.995536 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:06:24.995536 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:06:24.995536 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:06:24.996716 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:06:24.996716 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:06:24.996716 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:06:24.996716 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:06:24.996716 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:06:24.996716 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:06:24.996716 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:06:24.996716 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:06:24.996716 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:06:24.996716 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:06:24.996716 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:06:24.996716 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:06:24.996716 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:06:24.996716 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:06:24.996716 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:06:24.996716 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:06:24.996716 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:06:24.997641 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:06:24.997641 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:06:24.997641 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:06:24.997641 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 612/1242] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:06:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:06:25.002516 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fe0ba8-8b29-487d-97d0-04a34845cd28 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:25.106896 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:25.114378 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fe0ba8-8b29-487d-97d0-04a34845cd28 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:25.160839 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e9e91c-cdc5-47dd-9d8b-4d2ea4eb4cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:25.161464 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e9e91c-cdc5-47dd-9d8b-4d2ea4eb4cdc None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:25.161742 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e9e91c-cdc5-47dd-9d8b-4d2ea4eb4cdc None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:25.162020 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e9e91c-cdc5-47dd-9d8b-4d2ea4eb4cdc None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:25.162658 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e9e91c-cdc5-47dd-9d8b-4d2ea4eb4cdc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:06:25.162988 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e9e91c-cdc5-47dd-9d8b-4d2ea4eb4cdc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:25.164323 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e9e91c-cdc5-47dd-9d8b-4d2ea4eb4cdc None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:25.177264 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 631/1243] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:06:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:06:25.183237 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3753944d-b826-450b-86b0-6a17b5777f92 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:25.274319 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:25.302264 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3753944d-b826-450b-86b0-6a17b5777f92 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:25.343937 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38826ee3-1147-476e-90b1-63417f6c1782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:25.344573 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38826ee3-1147-476e-90b1-63417f6c1782 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:25.344902 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38826ee3-1147-476e-90b1-63417f6c1782 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:25.345206 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38826ee3-1147-476e-90b1-63417f6c1782 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:25.353411 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38826ee3-1147-476e-90b1-63417f6c1782 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:25.354863 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38826ee3-1147-476e-90b1-63417f6c1782 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:25.367066 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-38826ee3-1147-476e-90b1-63417f6c1782 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:06:25.367066 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:06:25.367066 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:06:25.367066 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:06:25.367066 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:06:25.367066 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:06:25.367066 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:06:25.367066 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:06:25.367066 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:06:25.367066 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:06:25.367066 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:06:25.367066 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:06:25.367066 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:06:25.367066 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:06:25.367066 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:06:25.367998 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:06:25.367998 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:06:25.367998 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:06:25.367998 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:06:25.367998 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:06:25.367998 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:06:25.367998 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:06:25.367998 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:06:25.367998 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:06:25.367998 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:06:25.367998 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:06:25.367998 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:06:25.367998 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:06:25.367998 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:06:25.367998 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:06:25.367998 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:06:25.367998 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:06:25.368959 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:06:25.368959 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:06:25.368959 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:06:25.368959 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:06:25.368959 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:06:25.368959 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:06:25.368959 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:06:25.368959 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:06:25.368959 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 613/1244] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 6 16:06:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:06:25.374643 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e9e91c-cdc5-47dd-9d8b-4d2ea4eb4cdc None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:25.485488 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e9e91c-cdc5-47dd-9d8b-4d2ea4eb4cdc None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:25.522128 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c919fcc-0487-4a15-8a04-97cf3a1951e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:25.522736 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c919fcc-0487-4a15-8a04-97cf3a1951e3 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:25.523010 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c919fcc-0487-4a15-8a04-97cf3a1951e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:25.523276 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c919fcc-0487-4a15-8a04-97cf3a1951e3 None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:25.523998 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c919fcc-0487-4a15-8a04-97cf3a1951e3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:06:25.524285 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c919fcc-0487-4a15-8a04-97cf3a1951e3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:25.525810 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c919fcc-0487-4a15-8a04-97cf3a1951e3 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:25.543177 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 632/1245] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 6 16:06:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:06:25.550097 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38826ee3-1147-476e-90b1-63417f6c1782 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:25.662559 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38826ee3-1147-476e-90b1-63417f6c1782 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:25.700532 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2d55a2-b42a-4fbe-b19f-f0289d1a0e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:25.701245 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2d55a2-b42a-4fbe-b19f-f0289d1a0e96 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:25.701538 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2d55a2-b42a-4fbe-b19f-f0289d1a0e96 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:25.701848 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2d55a2-b42a-4fbe-b19f-f0289d1a0e96 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:25.702683 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2d55a2-b42a-4fbe-b19f-f0289d1a0e96 None admin] RBAC: Adding query filter params (role.id=31321627b02c4ff188f8fe1521ce3181, scope.project.id=128d210e8dad48ac921ed4cb1d444ceb, user.id=71c2912a44f04bd983b4d4112dbee734) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:06:25.703002 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2d55a2-b42a-4fbe-b19f-f0289d1a0e96 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:25.704611 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2d55a2-b42a-4fbe-b19f-f0289d1a0e96 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:25.712223 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 614/1246] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 6 16:06:25 2022] GET /identity/v3/role_assignments?user.id=71c2912a44f04bd983b4d4112dbee734&scope.project.id=128d210e8dad48ac921ed4cb1d444ceb&role.id=31321627b02c4ff188f8fe1521ce3181 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:06:27.764594 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c91bac7-c501-4bee-adb9-f11dad139927 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:27.765359 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c91bac7-c501-4bee-adb9-f11dad139927 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:27.765359 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c91bac7-c501-4bee-adb9-f11dad139927 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:27.767130 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 633/1247] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:06:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:06:27.772728 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0674e28-6c15-4a3a-95be-6c36f6ff83f3 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:27.773173 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0674e28-6c15-4a3a-95be-6c36f6ff83f3 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:27.773465 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0674e28-6c15-4a3a-95be-6c36f6ff83f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:27.781361 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:27.796148 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-e0674e28-6c15-4a3a-95be-6c36f6ff83f3 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:06:27.797734 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e0674e28-6c15-4a3a-95be-6c36f6ff83f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:06:27.881293 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 615/1248] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:06:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 16:06:28.216861 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c91bac7-c501-4bee-adb9-f11dad139927 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:28.240510 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:28.344778 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c91bac7-c501-4bee-adb9-f11dad139927 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:28.383382 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01dde4e-a8b9-4461-9f12-5c45c551304f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:28.384149 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01dde4e-a8b9-4461-9f12-5c45c551304f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:28.384442 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01dde4e-a8b9-4461-9f12-5c45c551304f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:28.384698 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01dde4e-a8b9-4461-9f12-5c45c551304f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:28.396953 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01dde4e-a8b9-4461-9f12-5c45c551304f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:06:28.402161 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-c01dde4e-a8b9-4461-9f12-5c45c551304f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:06:28.402161 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:06:28.402161 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:06:28.402161 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:06:28.402161 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:06:28.402161 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:06:28.402161 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:06:28.402161 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:06:28.402161 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:06:28.402161 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:06:28.402161 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:06:28.402161 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:06:28.402161 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:06:28.402161 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:06:28.402161 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:06:28.402161 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:06:28.403228 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:06:28.403228 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:06:28.403228 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:06:28.403228 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:06:28.403228 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:06:28.403228 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:06:28.403228 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:06:28.403228 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:06:28.403228 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:06:28.403228 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:06:28.403228 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:06:28.403228 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:06:28.403228 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:06:28.403228 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:06:28.403228 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:06:28.403228 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:06:28.404074 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:06:28.404074 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:06:28.404074 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:06:28.404074 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:06:28.404074 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:06:28.404074 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:06:28.404074 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:06:28.404074 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:06:28.404074 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:06:28.404074 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:06:28.404074 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:06:28.404074 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:06:28.404074 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 634/1249] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 6 16:06:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:06:28.409916 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0674e28-6c15-4a3a-95be-6c36f6ff83f3 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:28.524741 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0674e28-6c15-4a3a-95be-6c36f6ff83f3 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:28.565064 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:28.571067 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334bdeeb-eb9e-4633-9a2d-d765ceedd5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:28.571706 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334bdeeb-eb9e-4633-9a2d-d765ceedd5c6 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:28.572048 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334bdeeb-eb9e-4633-9a2d-d765ceedd5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:28.572425 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334bdeeb-eb9e-4633-9a2d-d765ceedd5c6 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:28.573359 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334bdeeb-eb9e-4633-9a2d-d765ceedd5c6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:06:28.573858 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334bdeeb-eb9e-4633-9a2d-d765ceedd5c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:28.576071 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334bdeeb-eb9e-4633-9a2d-d765ceedd5c6 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:28.587037 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 616/1250] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 6 16:06:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:06:28.593274 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01dde4e-a8b9-4461-9f12-5c45c551304f None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:28.685892 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:28.715190 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01dde4e-a8b9-4461-9f12-5c45c551304f None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:28.757043 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a2ef30-86f9-4385-b36f-fbec0f752d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:28.757727 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a2ef30-86f9-4385-b36f-fbec0f752d29 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:28.758045 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a2ef30-86f9-4385-b36f-fbec0f752d29 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:28.758349 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a2ef30-86f9-4385-b36f-fbec0f752d29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:28.769678 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a2ef30-86f9-4385-b36f-fbec0f752d29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:28.771257 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a2ef30-86f9-4385-b36f-fbec0f752d29 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:28.786740 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-b9a2ef30-86f9-4385-b36f-fbec0f752d29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:06:28.786740 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:06:28.786740 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:06:28.786740 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:06:28.786740 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:06:28.786740 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:06:28.786740 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:06:28.786740 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:06:28.786740 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:06:28.786740 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:06:28.786740 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:06:28.786740 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:06:28.786740 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:06:28.786740 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:06:28.786740 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:06:28.786740 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:06:28.787877 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:06:28.787877 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:06:28.787877 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:06:28.787877 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:06:28.787877 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:06:28.787877 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:06:28.787877 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:06:28.787877 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:06:28.787877 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:06:28.787877 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:06:28.787877 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:06:28.787877 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:06:28.787877 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:06:28.787877 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:06:28.787877 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:06:28.787877 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:06:28.787877 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:06:28.788961 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:06:28.788961 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:06:28.788961 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:06:28.788961 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 635/1251] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:06:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:06:28.794594 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334bdeeb-eb9e-4633-9a2d-d765ceedd5c6 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:28.906169 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334bdeeb-eb9e-4633-9a2d-d765ceedd5c6 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:28.944627 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075243d2-94f7-498f-bc64-5f97b3d0ad2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:28.945328 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075243d2-94f7-498f-bc64-5f97b3d0ad2a None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:28.945824 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075243d2-94f7-498f-bc64-5f97b3d0ad2a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:28.946246 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075243d2-94f7-498f-bc64-5f97b3d0ad2a None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:28.946956 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075243d2-94f7-498f-bc64-5f97b3d0ad2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:06:28.947344 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075243d2-94f7-498f-bc64-5f97b3d0ad2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:28.948795 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075243d2-94f7-498f-bc64-5f97b3d0ad2a None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:28.962855 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 617/1252] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:06:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:06:28.968988 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a2ef30-86f9-4385-b36f-fbec0f752d29 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:29.084137 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a2ef30-86f9-4385-b36f-fbec0f752d29 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:29.113974 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:29.123137 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91f2cc2-4421-4235-bdbb-a1f52f171d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:29.123855 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f2cc2-4421-4235-bdbb-a1f52f171d97 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:29.124234 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f2cc2-4421-4235-bdbb-a1f52f171d97 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:29.124498 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f2cc2-4421-4235-bdbb-a1f52f171d97 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:29.133909 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91f2cc2-4421-4235-bdbb-a1f52f171d97 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:29.135568 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91f2cc2-4421-4235-bdbb-a1f52f171d97 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:29.147933 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-f91f2cc2-4421-4235-bdbb-a1f52f171d97 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 06 16:06:29.147933 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:06:29.147933 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:06:29.147933 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:06:29.147933 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:06:29.147933 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:06:29.147933 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:06:29.147933 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:06:29.147933 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:06:29.147933 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:06:29.147933 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:06:29.147933 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:06:29.147933 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:06:29.147933 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:06:29.147933 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:06:29.147933 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:06:29.149067 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:06:29.149067 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:06:29.149067 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:06:29.149067 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:06:29.149067 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:06:29.149067 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:06:29.149067 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:06:29.149067 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:06:29.149067 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:06:29.149067 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:06:29.149067 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:06:29.149067 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:06:29.149067 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:06:29.149067 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:06:29.149067 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:06:29.149067 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:06:29.149067 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:06:29.149067 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:06:29.150174 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:06:29.150174 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:06:29.150174 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:06:29.150174 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:06:29.150174 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:06:29.150174 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:06:29.150174 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:06:29.150174 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:06:29.150174 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 06 16:06:29.150174 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:06:29.150174 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 636/1253] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:06:28 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:06:29.155863 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075243d2-94f7-498f-bc64-5f97b3d0ad2a None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:29.216874 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:29.277217 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075243d2-94f7-498f-bc64-5f97b3d0ad2a None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:29.316487 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a993211a-e460-4244-ac27-82f4cc574d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:29.317135 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a993211a-e460-4244-ac27-82f4cc574d62 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:29.317418 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a993211a-e460-4244-ac27-82f4cc574d62 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:29.317689 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a993211a-e460-4244-ac27-82f4cc574d62 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:29.318465 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a993211a-e460-4244-ac27-82f4cc574d62 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:06:29.318731 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a993211a-e460-4244-ac27-82f4cc574d62 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:29.320252 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a993211a-e460-4244-ac27-82f4cc574d62 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:29.340935 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 618/1254] 10.222.0.43 () {56 vars in 1143 bytes} [Tue Sep 6 16:06:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:06:29.345448 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91f2cc2-4421-4235-bdbb-a1f52f171d97 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:29.471534 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91f2cc2-4421-4235-bdbb-a1f52f171d97 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:29.509432 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13005069-a256-4f98-8da6-8ff5606135dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:29.510147 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13005069-a256-4f98-8da6-8ff5606135dc None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:29.510412 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13005069-a256-4f98-8da6-8ff5606135dc None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:29.510687 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13005069-a256-4f98-8da6-8ff5606135dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:29.567851 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:29.581909 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13005069-a256-4f98-8da6-8ff5606135dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:29.583468 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13005069-a256-4f98-8da6-8ff5606135dc None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:29.703952 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 637/1255] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 6 16:06:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 16:06:29.711655 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a993211a-e460-4244-ac27-82f4cc574d62 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:29.837837 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a993211a-e460-4244-ac27-82f4cc574d62 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:29.858032 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:29.882268 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b679a5f8-5794-484b-b1bc-73d9849f0d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:29.882951 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b679a5f8-5794-484b-b1bc-73d9849f0d4e None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:29.883239 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b679a5f8-5794-484b-b1bc-73d9849f0d4e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:29.883503 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b679a5f8-5794-484b-b1bc-73d9849f0d4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:29.898458 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b679a5f8-5794-484b-b1bc-73d9849f0d4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:29.900225 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b679a5f8-5794-484b-b1bc-73d9849f0d4e None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:29.918936 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-b679a5f8-5794-484b-b1bc-73d9849f0d4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:06:29.918936 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:06:29.918936 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:06:29.918936 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:06:29.918936 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:06:29.918936 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:06:29.918936 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:06:29.918936 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:06:29.918936 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:06:29.918936 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:06:29.918936 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:06:29.918936 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:06:29.918936 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:06:29.918936 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:06:29.918936 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:06:29.918936 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:06:29.920134 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:06:29.920134 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:06:29.920134 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:06:29.920134 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:06:29.920134 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:06:29.920134 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:06:29.920134 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:06:29.920134 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:06:29.920134 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:06:29.920134 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:06:29.920134 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:06:29.920134 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:06:29.920134 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:06:29.920134 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:06:29.920134 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:06:29.920134 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:06:29.920134 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:06:29.921412 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:06:29.921412 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:06:29.921412 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:06:29.921412 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 619/1256] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:06:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:06:29.926847 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13005069-a256-4f98-8da6-8ff5606135dc None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:30.023140 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:30.037259 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13005069-a256-4f98-8da6-8ff5606135dc None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:30.072989 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c49ab01-f781-48fd-95a3-ae40ab9fafbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:30.073597 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c49ab01-f781-48fd-95a3-ae40ab9fafbc None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:30.073872 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c49ab01-f781-48fd-95a3-ae40ab9fafbc None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:30.074138 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c49ab01-f781-48fd-95a3-ae40ab9fafbc None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:30.075207 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c49ab01-f781-48fd-95a3-ae40ab9fafbc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:06:30.075503 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c49ab01-f781-48fd-95a3-ae40ab9fafbc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:30.076889 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c49ab01-f781-48fd-95a3-ae40ab9fafbc None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:30.087926 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 638/1257] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:06:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:06:30.093908 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b679a5f8-5794-484b-b1bc-73d9849f0d4e None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:30.218182 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b679a5f8-5794-484b-b1bc-73d9849f0d4e None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:30.259419 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc8764e-c0f1-4446-a597-411be7204de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:30.260062 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc8764e-c0f1-4446-a597-411be7204de7 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:30.260359 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc8764e-c0f1-4446-a597-411be7204de7 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:30.260617 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc8764e-c0f1-4446-a597-411be7204de7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:30.267802 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc8764e-c0f1-4446-a597-411be7204de7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:30.269305 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc8764e-c0f1-4446-a597-411be7204de7 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:30.282800 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-dbc8764e-c0f1-4446-a597-411be7204de7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:06:30.282800 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:06:30.282800 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:06:30.282800 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:06:30.282800 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:06:30.282800 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:06:30.282800 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:06:30.282800 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:06:30.282800 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:06:30.282800 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:06:30.282800 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:06:30.282800 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:06:30.282800 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:06:30.282800 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:06:30.282800 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:06:30.283867 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:06:30.283867 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:06:30.283867 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:06:30.283867 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:06:30.283867 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:06:30.283867 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:06:30.283867 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:06:30.283867 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:06:30.283867 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:06:30.283867 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:06:30.283867 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:06:30.283867 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:06:30.283867 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:06:30.283867 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:06:30.283867 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:06:30.283867 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:06:30.283867 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:06:30.285014 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:06:30.285014 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:06:30.285014 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:06:30.285014 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:06:30.285014 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:06:30.285014 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:06:30.285014 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:06:30.285014 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:06:30.285014 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 620/1258] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 6 16:06:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:06:30.290068 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c49ab01-f781-48fd-95a3-ae40ab9fafbc None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:30.396905 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c49ab01-f781-48fd-95a3-ae40ab9fafbc None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:30.434293 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966a522c-8f24-4cae-81da-004d51ba3e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:30.435129 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966a522c-8f24-4cae-81da-004d51ba3e3d None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:30.435546 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966a522c-8f24-4cae-81da-004d51ba3e3d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:30.435894 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966a522c-8f24-4cae-81da-004d51ba3e3d None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:30.436644 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966a522c-8f24-4cae-81da-004d51ba3e3d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:06:30.436977 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966a522c-8f24-4cae-81da-004d51ba3e3d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:30.438411 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966a522c-8f24-4cae-81da-004d51ba3e3d None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:30.451986 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 639/1259] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 6 16:06:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:06:30.458323 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc8764e-c0f1-4446-a597-411be7204de7 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:30.485776 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:30.564958 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc8764e-c0f1-4446-a597-411be7204de7 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:30.599946 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c541ba79-a508-4efe-b416-7ec026f1bb35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:30.600609 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c541ba79-a508-4efe-b416-7ec026f1bb35 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:30.600949 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c541ba79-a508-4efe-b416-7ec026f1bb35 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:30.601223 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c541ba79-a508-4efe-b416-7ec026f1bb35 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:30.602006 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c541ba79-a508-4efe-b416-7ec026f1bb35 None admin] RBAC: Adding query filter params (role.id=edaabdea8be84a7980fb2c651fe52f63, scope.project.id=128d210e8dad48ac921ed4cb1d444ceb, user.id=71c2912a44f04bd983b4d4112dbee734) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:06:30.602336 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c541ba79-a508-4efe-b416-7ec026f1bb35 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:30.603981 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c541ba79-a508-4efe-b416-7ec026f1bb35 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:30.610262 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 621/1260] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 6 16:06:30 2022] GET /identity/v3/role_assignments?user.id=71c2912a44f04bd983b4d4112dbee734&scope.project.id=128d210e8dad48ac921ed4cb1d444ceb&role.id=edaabdea8be84a7980fb2c651fe52f63 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:06:32.497340 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aee2584-1b8f-4b89-89dc-339b6817f7d7 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:32.498185 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aee2584-1b8f-4b89-89dc-339b6817f7d7 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:32.498185 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aee2584-1b8f-4b89-89dc-339b6817f7d7 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:32.500541 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 640/1261] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:06:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:06:32.506781 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557fb746-4396-490e-b231-df86564421ee None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:32.507504 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557fb746-4396-490e-b231-df86564421ee None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:32.507692 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557fb746-4396-490e-b231-df86564421ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:32.528683 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-557fb746-4396-490e-b231-df86564421ee None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:06:32.529842 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-557fb746-4396-490e-b231-df86564421ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:06:32.602662 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 622/1262] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:06:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:06:32.866856 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aee2584-1b8f-4b89-89dc-339b6817f7d7 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:32.880977 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:32.977161 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aee2584-1b8f-4b89-89dc-339b6817f7d7 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:33.013886 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbea6df0-cbe7-44cf-a79a-7c8b53baca6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:33.014495 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbea6df0-cbe7-44cf-a79a-7c8b53baca6c None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:33.014768 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbea6df0-cbe7-44cf-a79a-7c8b53baca6c None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:33.015035 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbea6df0-cbe7-44cf-a79a-7c8b53baca6c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:33.025617 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbea6df0-cbe7-44cf-a79a-7c8b53baca6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:06:33.030426 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-cbea6df0-cbe7-44cf-a79a-7c8b53baca6c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:06:33.030426 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:06:33.030426 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:06:33.030426 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:06:33.030426 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:06:33.030426 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:06:33.030426 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:06:33.030426 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:06:33.030426 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:06:33.030426 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:06:33.030426 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:06:33.030426 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:06:33.030426 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:06:33.030426 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:06:33.030426 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:06:33.030426 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:06:33.031473 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:06:33.031473 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:06:33.031473 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:06:33.031473 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:06:33.031473 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:06:33.031473 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:06:33.031473 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:06:33.031473 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:06:33.031473 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:06:33.031473 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:06:33.031473 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:06:33.031473 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:06:33.031473 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:06:33.031473 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:06:33.031473 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:06:33.031473 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:06:33.032328 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:06:33.032328 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:06:33.032328 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:06:33.032328 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:06:33.032328 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:06:33.032328 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:06:33.032328 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:06:33.032328 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:06:33.032328 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:06:33.032328 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:06:33.032328 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:06:33.032328 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:06:33.032328 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 641/1263] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 6 16:06:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:06:33.039959 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557fb746-4396-490e-b231-df86564421ee None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:33.060530 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:33.152214 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557fb746-4396-490e-b231-df86564421ee None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:33.187136 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cf6d0f-b95f-4898-a17f-aca233fe945a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:33.187730 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf6d0f-b95f-4898-a17f-aca233fe945a None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:33.188002 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf6d0f-b95f-4898-a17f-aca233fe945a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:33.188267 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf6d0f-b95f-4898-a17f-aca233fe945a None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:33.188958 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cf6d0f-b95f-4898-a17f-aca233fe945a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:06:33.189257 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cf6d0f-b95f-4898-a17f-aca233fe945a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:33.190635 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cf6d0f-b95f-4898-a17f-aca233fe945a None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:33.198693 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 623/1264] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 6 16:06:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:06:33.205622 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbea6df0-cbe7-44cf-a79a-7c8b53baca6c None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:33.284036 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:33.319742 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbea6df0-cbe7-44cf-a79a-7c8b53baca6c None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:33.356053 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91ea81a-742b-4e5d-a2bd-970a77d298e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:33.356680 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91ea81a-742b-4e5d-a2bd-970a77d298e2 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:33.356996 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91ea81a-742b-4e5d-a2bd-970a77d298e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:33.357268 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91ea81a-742b-4e5d-a2bd-970a77d298e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:33.369188 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91ea81a-742b-4e5d-a2bd-970a77d298e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:33.370668 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91ea81a-742b-4e5d-a2bd-970a77d298e2 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:33.387005 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-f91ea81a-742b-4e5d-a2bd-970a77d298e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:06:33.387005 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:06:33.387005 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:06:33.387005 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:06:33.387005 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:06:33.387005 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:06:33.387005 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:06:33.387005 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:06:33.387005 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:06:33.387005 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:06:33.387005 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:06:33.387005 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:06:33.387005 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:06:33.387005 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:06:33.387005 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:06:33.387005 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:06:33.388098 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:06:33.388098 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:06:33.388098 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:06:33.388098 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:06:33.388098 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:06:33.388098 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:06:33.388098 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:06:33.388098 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:06:33.388098 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:06:33.388098 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:06:33.388098 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:06:33.388098 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:06:33.388098 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:06:33.388098 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:06:33.388098 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:06:33.388098 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:06:33.388098 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:06:33.389135 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:06:33.389135 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:06:33.389135 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:06:33.389135 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 642/1265] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:06:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:06:33.394328 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cf6d0f-b95f-4898-a17f-aca233fe945a None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:33.480305 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:33.504880 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cf6d0f-b95f-4898-a17f-aca233fe945a None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:33.541904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eb1ab5-ebf6-431e-8cc3-72295d084cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:33.542495 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eb1ab5-ebf6-431e-8cc3-72295d084cef None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:33.542765 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eb1ab5-ebf6-431e-8cc3-72295d084cef None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:33.543038 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eb1ab5-ebf6-431e-8cc3-72295d084cef None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:33.543675 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eb1ab5-ebf6-431e-8cc3-72295d084cef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:06:33.543978 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eb1ab5-ebf6-431e-8cc3-72295d084cef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:33.545377 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eb1ab5-ebf6-431e-8cc3-72295d084cef None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:33.557083 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 624/1266] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:06:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:06:33.563184 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91ea81a-742b-4e5d-a2bd-970a77d298e2 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:33.670319 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91ea81a-742b-4e5d-a2bd-970a77d298e2 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:33.692564 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:33.715440 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb5f513-a7ac-4042-83dd-2c64faf30d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:33.716172 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb5f513-a7ac-4042-83dd-2c64faf30d6a None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:33.716448 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb5f513-a7ac-4042-83dd-2c64faf30d6a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:33.716878 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb5f513-a7ac-4042-83dd-2c64faf30d6a None admin] PATH_INFO: `/v3/users/placement` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:33.726115 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb5f513-a7ac-4042-83dd-2c64faf30d6a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:33.727624 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb5f513-a7ac-4042-83dd-2c64faf30d6a None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:33.743144 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-edb5f513-a7ac-4042-83dd-2c64faf30d6a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 06 16:06:33.743144 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:06:33.743144 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:06:33.743144 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:06:33.743144 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:06:33.743144 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:06:33.743144 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:06:33.743144 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:06:33.743144 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:06:33.743144 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:06:33.743144 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:06:33.743144 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:06:33.743144 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:06:33.743144 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:06:33.743144 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:06:33.743144 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:06:33.744073 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:06:33.744073 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:06:33.744073 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:06:33.744073 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:06:33.744073 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:06:33.744073 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:06:33.744073 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:06:33.744073 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:06:33.744073 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:06:33.744073 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:06:33.744073 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:06:33.744073 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:06:33.744073 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:06:33.744073 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:06:33.744073 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:06:33.744073 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:06:33.744073 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:06:33.744073 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:06:33.745249 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:06:33.745249 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:06:33.745249 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:06:33.745249 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:06:33.745249 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:06:33.745249 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:06:33.745249 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:06:33.745249 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:06:33.745249 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 06 16:06:33.745249 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:06:33.745249 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 643/1267] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:06:33 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:06:33.752390 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eb1ab5-ebf6-431e-8cc3-72295d084cef None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:33.856723 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eb1ab5-ebf6-431e-8cc3-72295d084cef None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:33.905417 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27304dd0-1a69-4499-98d7-91e4778d797d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:33.906022 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27304dd0-1a69-4499-98d7-91e4778d797d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:33.906296 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27304dd0-1a69-4499-98d7-91e4778d797d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:33.906566 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27304dd0-1a69-4499-98d7-91e4778d797d None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:33.907301 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27304dd0-1a69-4499-98d7-91e4778d797d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:06:33.907605 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27304dd0-1a69-4499-98d7-91e4778d797d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:33.909118 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27304dd0-1a69-4499-98d7-91e4778d797d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:33.918024 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:33.930716 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 625/1268] 10.222.0.43 () {56 vars in 1143 bytes} [Tue Sep 6 16:06:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:06:33.939035 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb5f513-a7ac-4042-83dd-2c64faf30d6a None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:34.051108 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb5f513-a7ac-4042-83dd-2c64faf30d6a None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:34.089792 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902a3388-24e3-40f1-a1a6-01c0b3c68ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:34.090432 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902a3388-24e3-40f1-a1a6-01c0b3c68ebd None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:34.090707 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902a3388-24e3-40f1-a1a6-01c0b3c68ebd None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:34.090977 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902a3388-24e3-40f1-a1a6-01c0b3c68ebd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:34.102067 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902a3388-24e3-40f1-a1a6-01c0b3c68ebd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:34.103520 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902a3388-24e3-40f1-a1a6-01c0b3c68ebd None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:34.117972 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-902a3388-24e3-40f1-a1a6-01c0b3c68ebd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:06:34.117972 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:06:34.117972 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:06:34.117972 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:06:34.117972 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:06:34.117972 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:06:34.117972 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:06:34.117972 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:06:34.117972 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:06:34.117972 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:06:34.117972 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:06:34.117972 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:06:34.117972 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:06:34.117972 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:06:34.117972 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:06:34.117972 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:06:34.119030 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:06:34.119030 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:06:34.119030 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:06:34.119030 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:06:34.119030 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:06:34.119030 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:06:34.119030 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:06:34.119030 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:06:34.119030 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:06:34.119030 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:06:34.119030 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:06:34.119030 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:06:34.119030 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:06:34.119030 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:06:34.119030 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:06:34.119030 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:06:34.119030 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:06:34.120261 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:06:34.120261 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:06:34.120261 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:06:34.120261 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 644/1269] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:06:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:06:34.125220 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27304dd0-1a69-4499-98d7-91e4778d797d None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:34.228558 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27304dd0-1a69-4499-98d7-91e4778d797d None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:34.268672 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4db0d3-326f-42ec-a261-b1ceb797cfec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:34.269317 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4db0d3-326f-42ec-a261-b1ceb797cfec None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:34.269589 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4db0d3-326f-42ec-a261-b1ceb797cfec None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:34.269858 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4db0d3-326f-42ec-a261-b1ceb797cfec None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:34.270502 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4db0d3-326f-42ec-a261-b1ceb797cfec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:06:34.270795 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4db0d3-326f-42ec-a261-b1ceb797cfec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:34.272175 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4db0d3-326f-42ec-a261-b1ceb797cfec None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:34.283811 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 626/1270] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:06:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:06:34.289849 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902a3388-24e3-40f1-a1a6-01c0b3c68ebd None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:34.323512 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:34.394427 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902a3388-24e3-40f1-a1a6-01c0b3c68ebd None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:34.428764 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8fd1b0-8b35-49a6-810c-c473eda72c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:34.429400 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8fd1b0-8b35-49a6-810c-c473eda72c7b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:34.429675 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8fd1b0-8b35-49a6-810c-c473eda72c7b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:34.429946 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8fd1b0-8b35-49a6-810c-c473eda72c7b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:34.437451 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8fd1b0-8b35-49a6-810c-c473eda72c7b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:34.438902 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8fd1b0-8b35-49a6-810c-c473eda72c7b None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:34.451238 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-ed8fd1b0-8b35-49a6-810c-c473eda72c7b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:06:34.451238 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:06:34.451238 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:06:34.451238 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:06:34.451238 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:06:34.451238 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:06:34.451238 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:06:34.451238 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:06:34.451238 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:06:34.451238 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:06:34.451238 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:06:34.451238 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:06:34.451238 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:06:34.451238 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:06:34.451238 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:06:34.452305 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:06:34.452305 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:06:34.452305 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:06:34.452305 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:06:34.452305 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:06:34.452305 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:06:34.452305 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:06:34.452305 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:06:34.452305 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:06:34.452305 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:06:34.452305 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:06:34.452305 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:06:34.452305 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:06:34.452305 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:06:34.452305 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:06:34.452305 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:06:34.452305 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:06:34.453509 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:06:34.453509 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:06:34.453509 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:06:34.453509 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:06:34.453509 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:06:34.453509 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:06:34.453509 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:06:34.453509 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:06:34.453509 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 645/1271] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 6 16:06:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:06:34.458308 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4db0d3-326f-42ec-a261-b1ceb797cfec None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:34.519242 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:34.561937 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4db0d3-326f-42ec-a261-b1ceb797cfec None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:34.602724 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5df6712-0750-4197-8cc0-9ecb4395dc48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:34.603244 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df6712-0750-4197-8cc0-9ecb4395dc48 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:34.603547 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df6712-0750-4197-8cc0-9ecb4395dc48 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:34.603811 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df6712-0750-4197-8cc0-9ecb4395dc48 None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:34.604511 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5df6712-0750-4197-8cc0-9ecb4395dc48 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:06:34.604821 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5df6712-0750-4197-8cc0-9ecb4395dc48 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:34.606397 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5df6712-0750-4197-8cc0-9ecb4395dc48 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:34.618515 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 627/1272] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 6 16:06:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:06:34.627460 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8fd1b0-8b35-49a6-810c-c473eda72c7b None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:34.727739 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:34.739048 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8fd1b0-8b35-49a6-810c-c473eda72c7b None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:34.775649 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6555bb-3b78-4fe6-b269-fa64216b19b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:34.776305 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6555bb-3b78-4fe6-b269-fa64216b19b1 None admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:34.776586 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6555bb-3b78-4fe6-b269-fa64216b19b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:34.776886 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6555bb-3b78-4fe6-b269-fa64216b19b1 None admin] PATH_INFO: `/v3/projects/128d210e8dad48ac921ed4cb1d444ceb/users/71c2912a44f04bd983b4d4112dbee734/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:34.778923 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6555bb-3b78-4fe6-b269-fa64216b19b1 None admin] RBAC: Authorizing `identity:create_grant(project_id=128d210e8dad48ac921ed4cb1d444ceb, user_id=71c2912a44f04bd983b4d4112dbee734, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:34.780390 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6555bb-3b78-4fe6-b269-fa64216b19b1 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:34.794826 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 646/1273] 10.222.0.43 () {58 vars in 1439 bytes} [Tue Sep 6 16:06:34 2022] PUT /identity/v3/projects/128d210e8dad48ac921ed4cb1d444ceb/users/71c2912a44f04bd983b4d4112dbee734/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:06:36.684825 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297d7588-82ce-4301-ac7a-45ae699db856 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:36.685672 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297d7588-82ce-4301-ac7a-45ae699db856 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:36.685672 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297d7588-82ce-4301-ac7a-45ae699db856 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:36.687801 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 628/1274] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:06:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:06:36.694000 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fc3c8d-3f8c-4641-98f0-aa759a8d3f45 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:36.694711 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fc3c8d-3f8c-4641-98f0-aa759a8d3f45 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:36.694862 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fc3c8d-3f8c-4641-98f0-aa759a8d3f45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:36.721390 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-c6fc3c8d-3f8c-4641-98f0-aa759a8d3f45 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:06:36.722667 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c6fc3c8d-3f8c-4641-98f0-aa759a8d3f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:06:36.795381 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 647/1275] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:06:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:06:37.061523 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297d7588-82ce-4301-ac7a-45ae699db856 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:37.178045 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297d7588-82ce-4301-ac7a-45ae699db856 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:37.195248 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:37.213966 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaf28c2-7d1c-4830-9b22-9d3df6576378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:37.214576 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaf28c2-7d1c-4830-9b22-9d3df6576378 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:37.214904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaf28c2-7d1c-4830-9b22-9d3df6576378 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:37.215174 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaf28c2-7d1c-4830-9b22-9d3df6576378 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:37.225522 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaf28c2-7d1c-4830-9b22-9d3df6576378 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:06:37.230371 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-ffaf28c2-7d1c-4830-9b22-9d3df6576378 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:06:37.230371 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:06:37.230371 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:06:37.230371 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:06:37.230371 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:06:37.230371 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:06:37.230371 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:06:37.230371 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:06:37.230371 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:06:37.230371 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:06:37.230371 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:06:37.230371 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:06:37.230371 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:06:37.230371 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:06:37.230371 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:06:37.230371 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:06:37.232806 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:06:37.232806 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:06:37.232806 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:06:37.232806 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:06:37.232806 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:06:37.232806 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:06:37.232806 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:06:37.232806 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:06:37.232806 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:06:37.232806 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:06:37.232806 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:06:37.232806 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:06:37.232806 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:06:37.232806 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:06:37.232806 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:06:37.232806 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:06:37.233937 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:06:37.233937 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:06:37.233937 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:06:37.233937 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:06:37.233937 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:06:37.233937 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:06:37.233937 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:06:37.233937 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:06:37.233937 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:06:37.233937 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:06:37.233937 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:06:37.233937 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:06:37.233937 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 629/1276] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 6 16:06:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:06:37.238857 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fc3c8d-3f8c-4641-98f0-aa759a8d3f45 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:37.323669 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:37.344189 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fc3c8d-3f8c-4641-98f0-aa759a8d3f45 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:37.377461 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3932da20-a272-4571-9235-1d31aba41551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:37.378053 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3932da20-a272-4571-9235-1d31aba41551 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:37.378329 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3932da20-a272-4571-9235-1d31aba41551 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:37.378596 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3932da20-a272-4571-9235-1d31aba41551 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:37.379254 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3932da20-a272-4571-9235-1d31aba41551 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:06:37.379549 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3932da20-a272-4571-9235-1d31aba41551 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:37.380977 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3932da20-a272-4571-9235-1d31aba41551 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:37.388739 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 648/1277] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 6 16:06:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:06:37.397349 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaf28c2-7d1c-4830-9b22-9d3df6576378 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:37.500808 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaf28c2-7d1c-4830-9b22-9d3df6576378 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:37.535769 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57da13af-aba0-4e27-8213-b400dba77c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:37.536389 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da13af-aba0-4e27-8213-b400dba77c64 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:37.536664 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da13af-aba0-4e27-8213-b400dba77c64 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:37.536962 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da13af-aba0-4e27-8213-b400dba77c64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:37.547801 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57da13af-aba0-4e27-8213-b400dba77c64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:37.549303 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57da13af-aba0-4e27-8213-b400dba77c64 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:37.563459 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-57da13af-aba0-4e27-8213-b400dba77c64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:06:37.563459 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:06:37.563459 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:06:37.563459 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:06:37.563459 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:06:37.563459 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:06:37.563459 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:06:37.563459 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:06:37.563459 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:06:37.563459 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:06:37.563459 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:06:37.563459 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:06:37.563459 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:06:37.563459 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:06:37.563459 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:06:37.563459 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:06:37.564609 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:06:37.564609 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:06:37.564609 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:06:37.564609 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:06:37.564609 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:06:37.564609 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:06:37.564609 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:06:37.564609 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:06:37.564609 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:06:37.564609 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:06:37.564609 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:06:37.564609 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:06:37.564609 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:06:37.564609 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:06:37.564609 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:06:37.564609 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:06:37.564609 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:06:37.565693 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:06:37.565693 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:06:37.565693 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:06:37.565693 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 630/1278] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:06:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:06:37.570586 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3932da20-a272-4571-9235-1d31aba41551 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:37.670015 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3932da20-a272-4571-9235-1d31aba41551 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:37.706073 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f7f0c-b76e-4184-99a3-4484513cd66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:37.706787 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f7f0c-b76e-4184-99a3-4484513cd66a None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:37.707136 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f7f0c-b76e-4184-99a3-4484513cd66a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:37.707405 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f7f0c-b76e-4184-99a3-4484513cd66a None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:37.708338 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528f7f0c-b76e-4184-99a3-4484513cd66a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:06:37.708695 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528f7f0c-b76e-4184-99a3-4484513cd66a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:37.710540 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528f7f0c-b76e-4184-99a3-4484513cd66a None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:37.726182 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 649/1279] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:06:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:06:37.732899 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57da13af-aba0-4e27-8213-b400dba77c64 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:37.768529 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:37.852592 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57da13af-aba0-4e27-8213-b400dba77c64 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:37.889676 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fde825-2995-4b62-9002-634cfe2445a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:37.890299 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fde825-2995-4b62-9002-634cfe2445a8 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:37.890575 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fde825-2995-4b62-9002-634cfe2445a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:37.890848 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fde825-2995-4b62-9002-634cfe2445a8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:37.898403 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fde825-2995-4b62-9002-634cfe2445a8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:37.899857 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fde825-2995-4b62-9002-634cfe2445a8 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:37.911253 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-87fde825-2995-4b62-9002-634cfe2445a8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 06 16:06:37.911253 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:06:37.911253 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:06:37.911253 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:06:37.911253 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:06:37.911253 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:06:37.911253 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:06:37.911253 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:06:37.911253 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:06:37.911253 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:06:37.911253 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:06:37.911253 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:06:37.911253 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:06:37.911253 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:06:37.911253 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:06:37.911253 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:06:37.912310 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:06:37.912310 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:06:37.912310 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:06:37.912310 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:06:37.912310 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:06:37.912310 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:06:37.912310 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:06:37.912310 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:06:37.912310 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:06:37.912310 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:06:37.912310 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:06:37.912310 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:06:37.912310 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:06:37.912310 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:06:37.912310 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:06:37.912310 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:06:37.912310 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:06:37.912310 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:06:37.913352 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:06:37.913352 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:06:37.913352 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:06:37.913352 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:06:37.913352 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:06:37.913352 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:06:37.913352 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:06:37.913352 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:06:37.913352 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 06 16:06:37.913352 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:06:37.913352 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 631/1280] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:06:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:06:37.918597 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f7f0c-b76e-4184-99a3-4484513cd66a None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:37.922655 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:38.021709 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f7f0c-b76e-4184-99a3-4484513cd66a None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:38.056525 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84357801-8fd0-4867-aca4-8b2559453c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:38.057145 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84357801-8fd0-4867-aca4-8b2559453c45 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:38.057420 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84357801-8fd0-4867-aca4-8b2559453c45 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:38.057684 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84357801-8fd0-4867-aca4-8b2559453c45 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:38.058406 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84357801-8fd0-4867-aca4-8b2559453c45 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:06:38.058753 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84357801-8fd0-4867-aca4-8b2559453c45 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:38.060183 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84357801-8fd0-4867-aca4-8b2559453c45 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:38.078649 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 650/1281] 10.222.0.43 () {56 vars in 1143 bytes} [Tue Sep 6 16:06:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:06:38.088235 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fde825-2995-4b62-9002-634cfe2445a8 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:38.181232 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:38.191779 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fde825-2995-4b62-9002-634cfe2445a8 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:38.227453 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e5717b-1f71-4756-82e1-5c3607346e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:38.228042 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e5717b-1f71-4756-82e1-5c3607346e90 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:38.228321 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e5717b-1f71-4756-82e1-5c3607346e90 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:38.228587 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e5717b-1f71-4756-82e1-5c3607346e90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:38.302590 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e5717b-1f71-4756-82e1-5c3607346e90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:38.304058 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e5717b-1f71-4756-82e1-5c3607346e90 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:38.422476 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:38.434383 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 632/1282] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 6 16:06:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 16:06:38.440487 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84357801-8fd0-4867-aca4-8b2559453c45 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:38.503683 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:38.543077 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84357801-8fd0-4867-aca4-8b2559453c45 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:38.578834 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a592299a-0af8-4cfd-aa81-efc9f9f97374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:38.579460 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a592299a-0af8-4cfd-aa81-efc9f9f97374 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:38.579735 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a592299a-0af8-4cfd-aa81-efc9f9f97374 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:38.580004 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a592299a-0af8-4cfd-aa81-efc9f9f97374 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:38.591199 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a592299a-0af8-4cfd-aa81-efc9f9f97374 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:38.592637 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a592299a-0af8-4cfd-aa81-efc9f9f97374 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:38.608120 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-a592299a-0af8-4cfd-aa81-efc9f9f97374 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:06:38.608120 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:06:38.608120 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:06:38.608120 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:06:38.608120 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:06:38.608120 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:06:38.608120 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:06:38.608120 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:06:38.608120 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:06:38.608120 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:06:38.608120 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:06:38.608120 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:06:38.608120 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:06:38.608120 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:06:38.608120 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:06:38.608120 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:06:38.609235 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:06:38.609235 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:06:38.609235 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:06:38.609235 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:06:38.609235 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:06:38.609235 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:06:38.609235 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:06:38.609235 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:06:38.609235 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:06:38.609235 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:06:38.609235 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:06:38.609235 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:06:38.609235 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:06:38.609235 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:06:38.609235 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:06:38.609235 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:06:38.609235 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:06:38.610852 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:06:38.610852 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:06:38.610852 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:06:38.610852 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 651/1283] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:06:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:06:38.618203 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e5717b-1f71-4756-82e1-5c3607346e90 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:38.730595 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e5717b-1f71-4756-82e1-5c3607346e90 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:38.765914 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b5fa4f-a482-4f86-a347-61d65d6ad26e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:38.766551 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b5fa4f-a482-4f86-a347-61d65d6ad26e None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:38.766769 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b5fa4f-a482-4f86-a347-61d65d6ad26e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:38.767035 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b5fa4f-a482-4f86-a347-61d65d6ad26e None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:38.767680 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b5fa4f-a482-4f86-a347-61d65d6ad26e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:06:38.767977 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b5fa4f-a482-4f86-a347-61d65d6ad26e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:38.769394 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b5fa4f-a482-4f86-a347-61d65d6ad26e None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:38.780815 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 633/1284] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 6 16:06:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:06:38.789841 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a592299a-0af8-4cfd-aa81-efc9f9f97374 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:38.897400 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a592299a-0af8-4cfd-aa81-efc9f9f97374 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:38.910689 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:38.932296 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967ed41f-4f49-43a5-9b70-66a6c1704b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:38.932935 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967ed41f-4f49-43a5-9b70-66a6c1704b90 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:38.933206 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967ed41f-4f49-43a5-9b70-66a6c1704b90 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:38.933514 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967ed41f-4f49-43a5-9b70-66a6c1704b90 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:38.940130 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967ed41f-4f49-43a5-9b70-66a6c1704b90 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:38.941638 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967ed41f-4f49-43a5-9b70-66a6c1704b90 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:38.953041 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-967ed41f-4f49-43a5-9b70-66a6c1704b90 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:06:38.953041 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:06:38.953041 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:06:38.953041 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:06:38.953041 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:06:38.953041 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:06:38.953041 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:06:38.953041 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:06:38.953041 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:06:38.953041 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:06:38.953041 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:06:38.953041 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:06:38.953041 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:06:38.953041 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:06:38.953041 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:06:38.954054 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:06:38.954054 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:06:38.954054 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:06:38.954054 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:06:38.954054 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:06:38.954054 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:06:38.954054 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:06:38.954054 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:06:38.954054 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 16:06:38.954054 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:06:38.954054 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:06:38.954054 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:06:38.954054 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:06:38.954054 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:06:38.954054 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:06:38.954054 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:06:38.954054 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:06:38.955073 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:06:38.955073 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:06:38.955073 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:06:38.955073 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:06:38.955073 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:06:38.955073 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:06:38.955073 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:06:38.955073 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 16:06:38.955073 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 652/1285] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 6 16:06:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:06:38.960605 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b5fa4f-a482-4f86-a347-61d65d6ad26e None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:39.015413 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:39.068745 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b5fa4f-a482-4f86-a347-61d65d6ad26e None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:39.104573 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bb1555-7811-4060-b2b1-2f7329e0b6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:39.105203 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bb1555-7811-4060-b2b1-2f7329e0b6c7 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:39.105532 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bb1555-7811-4060-b2b1-2f7329e0b6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:39.105803 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bb1555-7811-4060-b2b1-2f7329e0b6c7 None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:39.106487 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bb1555-7811-4060-b2b1-2f7329e0b6c7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:06:39.106788 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bb1555-7811-4060-b2b1-2f7329e0b6c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:39.108234 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bb1555-7811-4060-b2b1-2f7329e0b6c7 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:39.121094 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 634/1286] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 6 16:06:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:06:39.130966 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967ed41f-4f49-43a5-9b70-66a6c1704b90 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:39.237760 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967ed41f-4f49-43a5-9b70-66a6c1704b90 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:39.273409 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4baf83-02dc-4593-9ec9-451cc4705533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:39.273929 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4baf83-02dc-4593-9ec9-451cc4705533 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:39.274202 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4baf83-02dc-4593-9ec9-451cc4705533 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:39.274469 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4baf83-02dc-4593-9ec9-451cc4705533 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:39.275173 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4baf83-02dc-4593-9ec9-451cc4705533 None admin] RBAC: Adding query filter params (role.id=edaabdea8be84a7980fb2c651fe52f63, scope.project.id=128d210e8dad48ac921ed4cb1d444ceb, user.id=71c2912a44f04bd983b4d4112dbee734) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:06:39.275475 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4baf83-02dc-4593-9ec9-451cc4705533 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:39.276921 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4baf83-02dc-4593-9ec9-451cc4705533 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:39.282808 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 653/1287] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 6 16:06:39 2022] GET /identity/v3/role_assignments?user.id=71c2912a44f04bd983b4d4112dbee734&scope.project.id=128d210e8dad48ac921ed4cb1d444ceb&role.id=edaabdea8be84a7980fb2c651fe52f63 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:06:41.189351 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161227a3-2a27-483c-b76e-e2eccbc8766a None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:41.190270 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161227a3-2a27-483c-b76e-e2eccbc8766a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:41.190270 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161227a3-2a27-483c-b76e-e2eccbc8766a None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:41.192332 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 635/1288] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:06:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:06:41.199200 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba78c89-a480-49f5-acdf-0f64984e2cd8 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:41.200182 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba78c89-a480-49f5-acdf-0f64984e2cd8 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:41.200182 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba78c89-a480-49f5-acdf-0f64984e2cd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:41.220057 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-4ba78c89-a480-49f5-acdf-0f64984e2cd8 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:06:41.221298 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-4ba78c89-a480-49f5-acdf-0f64984e2cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:06:41.252177 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:41.290552 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 654/1289] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:06:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 16:06:41.554353 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161227a3-2a27-483c-b76e-e2eccbc8766a None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:41.668076 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161227a3-2a27-483c-b76e-e2eccbc8766a None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:41.680086 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:41.709616 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81380609-4d10-485e-8652-2e42d5f277d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:41.710735 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81380609-4d10-485e-8652-2e42d5f277d1 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:41.711233 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81380609-4d10-485e-8652-2e42d5f277d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:41.711644 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81380609-4d10-485e-8652-2e42d5f277d1 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:41.712414 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81380609-4d10-485e-8652-2e42d5f277d1 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:41.715222 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81380609-4d10-485e-8652-2e42d5f277d1 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:41.727860 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-81380609-4d10-485e-8652-2e42d5f277d1 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 16:06:41.727860 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:06:41.727860 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 16:06:41.727860 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 16:06:41.727860 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 16:06:41.727860 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 16:06:41.727860 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 16:06:41.727860 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:06:41.727860 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 16:06:41.727860 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:06:41.727860 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 16:06:41.727860 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:06:41.727860 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:06:41.727860 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:06:41.727860 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:06:41.727860 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:06:41.727860 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:06:41.729051 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:06:41.729051 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:06:41.729051 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:06:41.729051 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:06:41.729051 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:06:41.729051 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:06:41.729051 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 16:06:41.729051 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 16:06:41.729051 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 16:06:41.729051 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 16:06:41.729051 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:06:41.729051 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:06:41.729051 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:06:41.729051 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:06:41.729051 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:06:41.729051 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:06:41.730424 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:06:41.730424 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:06:41.730424 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:06:41.730424 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:06:41.730424 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:06:41.730424 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:06:41.730424 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:06:41.730424 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:06:41.730424 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:06:41.730424 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:06:41.730424 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 16:06:41.730424 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:06:41.730424 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 16:06:41.730424 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:06:41.730424 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 636/1290] 10.222.0.43 () {56 vars in 1117 bytes} [Tue Sep 6 16:06:41 2022] GET /identity/v3/services/placement => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:06:41.737251 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba78c89-a480-49f5-acdf-0f64984e2cd8 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:41.855933 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba78c89-a480-49f5-acdf-0f64984e2cd8 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:41.892012 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c84b8bd-8d68-4797-aa08-7dcbb8405ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:41.892604 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c84b8bd-8d68-4797-aa08-7dcbb8405ab8 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:41.892907 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c84b8bd-8d68-4797-aa08-7dcbb8405ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:41.893171 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c84b8bd-8d68-4797-aa08-7dcbb8405ab8 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:41.893791 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c84b8bd-8d68-4797-aa08-7dcbb8405ab8 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:06:41.894088 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c84b8bd-8d68-4797-aa08-7dcbb8405ab8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:41.895453 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c84b8bd-8d68-4797-aa08-7dcbb8405ab8 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:41.900285 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 655/1291] 10.222.0.43 () {56 vars in 1116 bytes} [Tue Sep 6 16:06:41 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:06:41.906512 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81380609-4d10-485e-8652-2e42d5f277d1 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:42.024643 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81380609-4d10-485e-8652-2e42d5f277d1 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:42.066866 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c514b6e-61c4-4250-8f13-ca76efe3a5fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:42.067452 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c514b6e-61c4-4250-8f13-ca76efe3a5fb None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:42.067726 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c514b6e-61c4-4250-8f13-ca76efe3a5fb None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:42.067993 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c514b6e-61c4-4250-8f13-ca76efe3a5fb None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:42.068616 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c514b6e-61c4-4250-8f13-ca76efe3a5fb None admin] RBAC: Adding query filter params (type=placement) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:06:42.068977 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c514b6e-61c4-4250-8f13-ca76efe3a5fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:42.070340 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c514b6e-61c4-4250-8f13-ca76efe3a5fb None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:42.075345 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 637/1292] 10.222.0.43 () {56 vars in 1116 bytes} [Tue Sep 6 16:06:41 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:06:43.952481 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4598643-bda9-413d-ac0f-1e12a0643ed8 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:43.953407 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4598643-bda9-413d-ac0f-1e12a0643ed8 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:43.953407 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4598643-bda9-413d-ac0f-1e12a0643ed8 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:43.955593 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 656/1293] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:06:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:06:43.961762 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d817db44-97b9-4ae5-adeb-1fc49e792735 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:43.962214 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d817db44-97b9-4ae5-adeb-1fc49e792735 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:43.962317 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d817db44-97b9-4ae5-adeb-1fc49e792735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:43.982011 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-d817db44-97b9-4ae5-adeb-1fc49e792735 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:06:43.983206 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d817db44-97b9-4ae5-adeb-1fc49e792735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:06:44.014542 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:44.056894 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 638/1294] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:06:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:06:44.323255 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4598643-bda9-413d-ac0f-1e12a0643ed8 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:44.359888 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:44.444846 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4598643-bda9-413d-ac0f-1e12a0643ed8 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:44.479891 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28192208-d578-4c2a-8129-4ca8485c31c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:44.480528 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28192208-d578-4c2a-8129-4ca8485c31c2 None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:44.480797 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28192208-d578-4c2a-8129-4ca8485c31c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:44.481224 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28192208-d578-4c2a-8129-4ca8485c31c2 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:44.481787 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28192208-d578-4c2a-8129-4ca8485c31c2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:44.483149 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28192208-d578-4c2a-8129-4ca8485c31c2 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:44.500385 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 657/1295] 10.222.0.43 () {60 vars in 1141 bytes} [Tue Sep 6 16:06:44 2022] POST /identity/v3/services => generated 246 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:06:46.455688 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc4f41d-16a4-453c-ae32-7729d2050d09 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:46.456247 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc4f41d-16a4-453c-ae32-7729d2050d09 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:46.456247 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc4f41d-16a4-453c-ae32-7729d2050d09 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:46.457603 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 639/1296] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:06:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:06:46.463897 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d4130d-4528-4639-a7c3-62774fdf628d None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:46.464665 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d4130d-4528-4639-a7c3-62774fdf628d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:46.464787 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d4130d-4528-4639-a7c3-62774fdf628d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:46.485391 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-83d4130d-4528-4639-a7c3-62774fdf628d None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:06:46.486583 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-83d4130d-4528-4639-a7c3-62774fdf628d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:06:46.564464 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 658/1297] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:06:46 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:06:46.846044 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc4f41d-16a4-453c-ae32-7729d2050d09 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:46.963493 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc4f41d-16a4-453c-ae32-7729d2050d09 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:47.008073 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8843313-0f8a-4958-b6bd-14929cd869ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:47.008693 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8843313-0f8a-4958-b6bd-14929cd869ba None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:47.008994 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8843313-0f8a-4958-b6bd-14929cd869ba None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:47.009262 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8843313-0f8a-4958-b6bd-14929cd869ba None admin] PATH_INFO: `/v3/services/placement` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:47.009841 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8843313-0f8a-4958-b6bd-14929cd869ba None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:47.011269 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8843313-0f8a-4958-b6bd-14929cd869ba None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:47.020380 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-d8843313-0f8a-4958-b6bd-14929cd869ba None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 16:06:47.020380 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:06:47.020380 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 16:06:47.020380 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 16:06:47.020380 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 16:06:47.020380 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 16:06:47.020380 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 16:06:47.020380 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:06:47.020380 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 16:06:47.020380 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:06:47.020380 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 16:06:47.020380 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:06:47.020380 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:06:47.020380 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:06:47.020380 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:06:47.020380 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:06:47.020380 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:06:47.021546 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:06:47.021546 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:06:47.021546 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:06:47.021546 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:06:47.021546 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:06:47.021546 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:06:47.021546 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 16:06:47.021546 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 16:06:47.021546 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 16:06:47.021546 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 16:06:47.021546 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:06:47.021546 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:06:47.021546 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:06:47.021546 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:06:47.021546 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:06:47.021546 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:06:47.022561 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:06:47.022561 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:06:47.022561 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:06:47.022561 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:06:47.022561 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:06:47.022561 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:06:47.022561 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:06:47.022561 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:06:47.022561 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:06:47.022561 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:06:47.022561 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 16:06:47.022561 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:06:47.022561 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 16:06:47.022561 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:06:47.022561 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 640/1298] 10.222.0.43 () {56 vars in 1117 bytes} [Tue Sep 6 16:06:46 2022] GET /identity/v3/services/placement => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:06:47.028105 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d4130d-4528-4639-a7c3-62774fdf628d None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:47.049973 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:47.136872 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d4130d-4528-4639-a7c3-62774fdf628d None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:47.172992 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c02732-baf1-454c-9c22-3d66d49b9c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:47.173588 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c02732-baf1-454c-9c22-3d66d49b9c79 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:47.173859 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c02732-baf1-454c-9c22-3d66d49b9c79 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:47.174123 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c02732-baf1-454c-9c22-3d66d49b9c79 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:47.174742 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c02732-baf1-454c-9c22-3d66d49b9c79 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:06:47.175033 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c02732-baf1-454c-9c22-3d66d49b9c79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:47.176404 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c02732-baf1-454c-9c22-3d66d49b9c79 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:47.181786 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 659/1299] 10.222.0.43 () {56 vars in 1116 bytes} [Tue Sep 6 16:06:47 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:06:47.188530 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8843313-0f8a-4958-b6bd-14929cd869ba None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:47.223493 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:47.299668 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8843313-0f8a-4958-b6bd-14929cd869ba None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:47.339169 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d062d7-a07c-442a-9732-47ecf99bf0a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:47.339784 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d062d7-a07c-442a-9732-47ecf99bf0a3 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:47.340059 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d062d7-a07c-442a-9732-47ecf99bf0a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:47.340321 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d062d7-a07c-442a-9732-47ecf99bf0a3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:47.341062 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d062d7-a07c-442a-9732-47ecf99bf0a3 None admin] RBAC: Adding query filter params (interface=public, service_id=fa989741a395487c9c08c9183a078669, region_id=RegionOne) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:06:47.341364 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d062d7-a07c-442a-9732-47ecf99bf0a3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:47.342916 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d062d7-a07c-442a-9732-47ecf99bf0a3 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:47.348745 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 641/1300] 10.222.0.43 () {56 vars in 1251 bytes} [Tue Sep 6 16:06:47 2022] GET /identity/v3/endpoints?service_id=fa989741a395487c9c08c9183a078669&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:06:47.355727 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c02732-baf1-454c-9c22-3d66d49b9c79 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:47.450276 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:47.471514 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c02732-baf1-454c-9c22-3d66d49b9c79 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:47.505326 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57e7310-8588-444f-a5a2-fec721c46ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:47.505911 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e7310-8588-444f-a5a2-fec721c46ee6 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:47.506186 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e7310-8588-444f-a5a2-fec721c46ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:47.506449 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e7310-8588-444f-a5a2-fec721c46ee6 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:47.507039 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57e7310-8588-444f-a5a2-fec721c46ee6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:47.508398 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57e7310-8588-444f-a5a2-fec721c46ee6 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:47.515236 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 660/1301] 10.222.0.43 () {56 vars in 1087 bytes} [Tue Sep 6 16:06:47 2022] GET /identity/v3/services => generated 2191 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:06:49.398142 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8cf30c-681f-42b4-8e65-74b3c4ffee62 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:49.398965 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8cf30c-681f-42b4-8e65-74b3c4ffee62 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:49.398965 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8cf30c-681f-42b4-8e65-74b3c4ffee62 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:49.401202 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 642/1302] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:06:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:06:49.407426 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf3dbaa-f5ca-404a-b250-6025cbecda81 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:49.407877 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf3dbaa-f5ca-404a-b250-6025cbecda81 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:49.407977 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf3dbaa-f5ca-404a-b250-6025cbecda81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:49.435704 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-7cf3dbaa-f5ca-404a-b250-6025cbecda81 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:06:49.436928 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7cf3dbaa-f5ca-404a-b250-6025cbecda81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:06:49.505574 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 661/1303] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:06:49 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:06:49.771727 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8cf30c-681f-42b4-8e65-74b3c4ffee62 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:49.898302 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:49.906221 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8cf30c-681f-42b4-8e65-74b3c4ffee62 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:49.940852 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11e9c13-b574-447e-a2e4-35e72c126ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:49.941473 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11e9c13-b574-447e-a2e4-35e72c126ee6 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:49.941748 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11e9c13-b574-447e-a2e4-35e72c126ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:49.941994 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11e9c13-b574-447e-a2e4-35e72c126ee6 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:49.942608 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11e9c13-b574-447e-a2e4-35e72c126ee6 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:49.943995 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11e9c13-b574-447e-a2e4-35e72c126ee6 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:49.953544 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-b11e9c13-b574-447e-a2e4-35e72c126ee6 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 16:06:49.953544 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:06:49.953544 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 16:06:49.953544 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 16:06:49.953544 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 16:06:49.953544 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 16:06:49.953544 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 16:06:49.953544 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:06:49.953544 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 16:06:49.953544 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:06:49.953544 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 16:06:49.953544 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:06:49.953544 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:06:49.953544 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:06:49.953544 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:06:49.953544 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:06:49.953544 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:06:49.954669 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:06:49.954669 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:06:49.954669 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:06:49.954669 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:06:49.954669 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:06:49.954669 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:06:49.954669 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 16:06:49.954669 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 16:06:49.954669 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 16:06:49.954669 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 16:06:49.954669 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:06:49.954669 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:06:49.954669 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:06:49.954669 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:06:49.954669 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:06:49.954669 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:06:49.955942 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:06:49.955942 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 16:06:49.955942 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:06:49.955942 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:06:49.955942 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:06:49.955942 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:06:49.955942 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:06:49.955942 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:06:49.955942 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:06:49.955942 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:06:49.955942 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 16:06:49.955942 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:06:49.955942 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 16:06:49.955942 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 16:06:49.955942 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 643/1304] 10.222.0.43 () {56 vars in 1117 bytes} [Tue Sep 6 16:06:49 2022] GET /identity/v3/services/placement => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:06:49.963413 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf3dbaa-f5ca-404a-b250-6025cbecda81 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:50.044136 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:50.067800 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf3dbaa-f5ca-404a-b250-6025cbecda81 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:50.103976 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a396f222-38da-46cf-b6ed-2199fc878160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:50.104576 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a396f222-38da-46cf-b6ed-2199fc878160 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:50.104902 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a396f222-38da-46cf-b6ed-2199fc878160 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:50.105172 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a396f222-38da-46cf-b6ed-2199fc878160 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:50.105795 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a396f222-38da-46cf-b6ed-2199fc878160 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:06:50.106092 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a396f222-38da-46cf-b6ed-2199fc878160 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:50.107464 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a396f222-38da-46cf-b6ed-2199fc878160 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:50.112794 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 662/1305] 10.222.0.43 () {56 vars in 1116 bytes} [Tue Sep 6 16:06:49 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:06:50.119498 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11e9c13-b574-447e-a2e4-35e72c126ee6 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:50.239002 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11e9c13-b574-447e-a2e4-35e72c126ee6 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:50.275508 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02875fd6-f5fa-4b5c-9a8e-7f6ff40df7ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:50.276119 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02875fd6-f5fa-4b5c-9a8e-7f6ff40df7ec None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:50.276856 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02875fd6-f5fa-4b5c-9a8e-7f6ff40df7ec None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:50.276856 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02875fd6-f5fa-4b5c-9a8e-7f6ff40df7ec None admin] PATH_INFO: `/v3/endpoints` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:50.277339 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02875fd6-f5fa-4b5c-9a8e-7f6ff40df7ec None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:50.278728 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02875fd6-f5fa-4b5c-9a8e-7f6ff40df7ec None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:50.300624 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 644/1306] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:06:50 2022] POST /identity/v3/endpoints => generated 331 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:07:00.109601 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=30 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:07:00.296191 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:07:54.148912 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944f70bb-6b3a-4afa-b6d6-e13ccf19bef8 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:07:54.149953 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944f70bb-6b3a-4afa-b6d6-e13ccf19bef8 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:07:54.150437 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944f70bb-6b3a-4afa-b6d6-e13ccf19bef8 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:07:54.157457 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17309cb8-f6e9-4259-93d2-1308b5fcb97e None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:07:54.158293 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17309cb8-f6e9-4259-93d2-1308b5fcb97e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:07:54.158794 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17309cb8-f6e9-4259-93d2-1308b5fcb97e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:07:54.161026 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 663/1307] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:07:54 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:07:54.238012 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-17309cb8-f6e9-4259-93d2-1308b5fcb97e None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:07:54.239788 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-17309cb8-f6e9-4259-93d2-1308b5fcb97e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:07:54.363092 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 645/1308] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:07:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:07:54.658663 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc9adc9-557d-40be-ad06-68a48e673d94 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:07:54.659299 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc9adc9-557d-40be-ad06-68a48e673d94 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:07:54.659777 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc9adc9-557d-40be-ad06-68a48e673d94 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:07:54.661424 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 664/1309] 10.222.0.43 () {54 vars in 965 bytes} [Tue Sep 6 16:07:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:07:54.668405 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d038a2-4e27-43dd-8cbc-e1d3e3f36225 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:07:54.668799 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d038a2-4e27-43dd-8cbc-e1d3e3f36225 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:07:54.669043 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d038a2-4e27-43dd-8cbc-e1d3e3f36225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:07:54.755472 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-d8d038a2-4e27-43dd-8cbc-e1d3e3f36225 None None] MFA Rules not processed for user `10565aefeaaa40549ed38bb9e0f4b189`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:07:54.757052 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d8d038a2-4e27-43dd-8cbc-e1d3e3f36225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:07:54.779673 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:07:54.840437 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 646/1310] 10.222.0.43 () {58 vars in 1062 bytes} [Tue Sep 6 16:07:54 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:07:54.849769 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc9adc9-557d-40be-ad06-68a48e673d94 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:07:54.851737 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5cc9adc9-557d-40be-ad06-68a48e673d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:07:54.977251 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc9adc9-557d-40be-ad06-68a48e673d94 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:07:55.057921 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79d741c-1a29-482f-8089-7588143762af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:07:55.059056 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79d741c-1a29-482f-8089-7588143762af service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:07:55.059479 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79d741c-1a29-482f-8089-7588143762af service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:07:55.059892 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79d741c-1a29-482f-8089-7588143762af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:07:55.061657 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e79d741c-1a29-482f-8089-7588143762af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:07:55.120078 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:07:55.180596 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79d741c-1a29-482f-8089-7588143762af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:07:55.183180 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79d741c-1a29-482f-8089-7588143762af service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:07:55.206952 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 665/1311] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:07:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:07:57.372982 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e5d94-4c63-454b-a28d-4009fe9798a5 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:07:57.373594 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e5d94-4c63-454b-a28d-4009fe9798a5 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:07:57.373594 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e5d94-4c63-454b-a28d-4009fe9798a5 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:07:57.375049 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 647/1312] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:07:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:07:57.380276 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a34520-fdc6-45cf-a7a1-0ab0079e79d1 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:07:57.380708 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a34520-fdc6-45cf-a7a1-0ab0079e79d1 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:07:57.380877 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a34520-fdc6-45cf-a7a1-0ab0079e79d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:07:57.467358 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-15a34520-fdc6-45cf-a7a1-0ab0079e79d1 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:07:57.468581 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-15a34520-fdc6-45cf-a7a1-0ab0079e79d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:07:57.581757 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 666/1313] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:07:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:07:57.861907 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93e5d94-4c63-454b-a28d-4009fe9798a5 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:07:57.863615 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f93e5d94-4c63-454b-a28d-4009fe9798a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:07:57.979477 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93e5d94-4c63-454b-a28d-4009fe9798a5 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:07:58.051296 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:07:58.054265 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f1cdf4-3595-4832-a2b7-29d885e18c10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:07:58.054923 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f1cdf4-3595-4832-a2b7-29d885e18c10 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:07:58.055351 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f1cdf4-3595-4832-a2b7-29d885e18c10 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:07:58.056104 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f1cdf4-3595-4832-a2b7-29d885e18c10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:07:58.058162 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-44f1cdf4-3595-4832-a2b7-29d885e18c10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:07:58.170406 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f1cdf4-3595-4832-a2b7-29d885e18c10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:07:58.171760 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f1cdf4-3595-4832-a2b7-29d885e18c10 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:07:58.190813 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 648/1314] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:07:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:08:00.415401 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798670bc-8daf-4fac-b097-a4e238d898d4 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:00.416291 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798670bc-8daf-4fac-b097-a4e238d898d4 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:00.416291 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798670bc-8daf-4fac-b097-a4e238d898d4 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:00.419477 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 667/1315] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:08:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:08:00.427508 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c785d83-ed25-46b4-9511-db64e0c2283b None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:00.428217 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c785d83-ed25-46b4-9511-db64e0c2283b None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:00.428405 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c785d83-ed25-46b4-9511-db64e0c2283b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:00.503754 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-0c785d83-ed25-46b4-9511-db64e0c2283b None None] MFA Rules not processed for user `6962122ffdbe4d6288f017e98d1e8ecb`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:08:00.505003 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0c785d83-ed25-46b4-9511-db64e0c2283b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:00.559703 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:00.605784 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 649/1316] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:08:00 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:08:00.892276 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798670bc-8daf-4fac-b097-a4e238d898d4 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:08:00.894854 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-798670bc-8daf-4fac-b097-a4e238d898d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:00.911278 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:01.026584 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798670bc-8daf-4fac-b097-a4e238d898d4 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:08:01.109209 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381743dc-53d5-4f62-bcb9-22b7f52f1c92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:08:01.109824 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381743dc-53d5-4f62-bcb9-22b7f52f1c92 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:01.110100 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381743dc-53d5-4f62-bcb9-22b7f52f1c92 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:01.110368 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381743dc-53d5-4f62-bcb9-22b7f52f1c92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:01.111985 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-381743dc-53d5-4f62-bcb9-22b7f52f1c92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:01.179964 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:01.229533 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381743dc-53d5-4f62-bcb9-22b7f52f1c92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:08:01.230995 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381743dc-53d5-4f62-bcb9-22b7f52f1c92 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:08:01.250513 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 668/1317] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:08:00 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:08:03.705222 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee43b8-63a6-4eae-ad74-2753b8f8411f None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:03.705754 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee43b8-63a6-4eae-ad74-2753b8f8411f None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:03.705754 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee43b8-63a6-4eae-ad74-2753b8f8411f None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:03.707046 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 650/1318] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:08:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:08:03.713961 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15bacf6-2877-4af4-ab9a-f5d433d6b8f2 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:03.714754 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15bacf6-2877-4af4-ab9a-f5d433d6b8f2 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:03.714937 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15bacf6-2877-4af4-ab9a-f5d433d6b8f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:03.791567 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-d15bacf6-2877-4af4-ab9a-f5d433d6b8f2 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:08:03.792843 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d15bacf6-2877-4af4-ab9a-f5d433d6b8f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:03.893435 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 669/1319] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:08:03 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:08:04.181977 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ee43b8-63a6-4eae-ad74-2753b8f8411f None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:08:04.184329 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-82ee43b8-63a6-4eae-ad74-2753b8f8411f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:04.310443 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ee43b8-63a6-4eae-ad74-2753b8f8411f None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:08:04.394661 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ba092b-a07b-45e5-80f4-beafe6860140 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:08:04.395274 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ba092b-a07b-45e5-80f4-beafe6860140 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:04.395554 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ba092b-a07b-45e5-80f4-beafe6860140 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:04.395821 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ba092b-a07b-45e5-80f4-beafe6860140 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:04.397444 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b5ba092b-a07b-45e5-80f4-beafe6860140 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:04.405934 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:04.517708 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ba092b-a07b-45e5-80f4-beafe6860140 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:08:04.519164 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ba092b-a07b-45e5-80f4-beafe6860140 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:08:04.538015 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 651/1320] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:08:04 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:08:07.086322 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11024166-9577-4257-a283-4ccc52097dcc None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:07.087020 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11024166-9577-4257-a283-4ccc52097dcc None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:07.087020 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11024166-9577-4257-a283-4ccc52097dcc None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:07.088235 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 670/1321] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:08:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:08:07.093676 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ebe733-ed52-4b01-89a0-688a1bb2cb0e None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:07.094151 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ebe733-ed52-4b01-89a0-688a1bb2cb0e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:07.094253 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ebe733-ed52-4b01-89a0-688a1bb2cb0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:07.165356 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-85ebe733-ed52-4b01-89a0-688a1bb2cb0e None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:08:07.166581 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-85ebe733-ed52-4b01-89a0-688a1bb2cb0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:07.226989 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:07.270335 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 652/1322] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:08:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:08:07.546173 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11024166-9577-4257-a283-4ccc52097dcc None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:08:07.547798 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-11024166-9577-4257-a283-4ccc52097dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:07.569169 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:07.658358 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11024166-9577-4257-a283-4ccc52097dcc None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:08:07.734660 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8263f422-f852-41ad-a9ab-bde17309a877 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:08:07.735389 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8263f422-f852-41ad-a9ab-bde17309a877 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:07.735671 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8263f422-f852-41ad-a9ab-bde17309a877 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:07.735937 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8263f422-f852-41ad-a9ab-bde17309a877 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:07.737562 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8263f422-f852-41ad-a9ab-bde17309a877 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:07.807324 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:07.846968 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8263f422-f852-41ad-a9ab-bde17309a877 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:08:07.848407 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8263f422-f852-41ad-a9ab-bde17309a877 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:08:07.867199 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 671/1323] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:08:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:08:09.785435 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d7590-0517-43e6-bfb4-58dec3af9f3a None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:09.786016 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d7590-0517-43e6-bfb4-58dec3af9f3a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:09.786016 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d7590-0517-43e6-bfb4-58dec3af9f3a None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:09.787321 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 653/1324] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:08:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:08:09.792591 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3856281b-d2f3-4143-8e08-2d956597aa8f None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:09.793140 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3856281b-d2f3-4143-8e08-2d956597aa8f None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:09.793253 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3856281b-d2f3-4143-8e08-2d956597aa8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:09.863711 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-3856281b-d2f3-4143-8e08-2d956597aa8f None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:08:09.864795 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-3856281b-d2f3-4143-8e08-2d956597aa8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:09.966811 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 672/1325] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:08:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:08:10.259723 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313d8312-abc1-4dac-8bce-2da71804e5a9 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:10.259981 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313d8312-abc1-4dac-8bce-2da71804e5a9 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:10.260252 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313d8312-abc1-4dac-8bce-2da71804e5a9 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:10.261655 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 654/1326] 10.222.0.43 () {54 vars in 965 bytes} [Tue Sep 6 16:08:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:08:10.267834 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e474aef-18e1-40fc-b372-9ef1aaef5ee2 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:10.268193 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e474aef-18e1-40fc-b372-9ef1aaef5ee2 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:10.268536 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e474aef-18e1-40fc-b372-9ef1aaef5ee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:10.303693 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:10.350714 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-9e474aef-18e1-40fc-b372-9ef1aaef5ee2 None None] MFA Rules not processed for user `10565aefeaaa40549ed38bb9e0f4b189`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:08:10.351929 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-9e474aef-18e1-40fc-b372-9ef1aaef5ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:10.440392 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 673/1327] 10.222.0.43 () {58 vars in 1062 bytes} [Tue Sep 6 16:08:10 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:08:10.450226 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313d8312-abc1-4dac-8bce-2da71804e5a9 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:08:10.451766 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-313d8312-abc1-4dac-8bce-2da71804e5a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:10.558992 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313d8312-abc1-4dac-8bce-2da71804e5a9 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:08:10.634115 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c37e54-c1f3-4546-9af1-e5e372ca21ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:08:10.634705 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c37e54-c1f3-4546-9af1-e5e372ca21ae service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:10.634975 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c37e54-c1f3-4546-9af1-e5e372ca21ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:10.635236 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c37e54-c1f3-4546-9af1-e5e372ca21ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:10.636963 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a7c37e54-c1f3-4546-9af1-e5e372ca21ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:10.653719 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:10.747557 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c37e54-c1f3-4546-9af1-e5e372ca21ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:08:10.749010 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c37e54-c1f3-4546-9af1-e5e372ca21ae service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:08:10.767037 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 655/1328] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:08:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:08:12.935089 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6977e88-903f-492b-b2ad-7b1dd1aa0250 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:12.935682 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6977e88-903f-492b-b2ad-7b1dd1aa0250 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:12.935682 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6977e88-903f-492b-b2ad-7b1dd1aa0250 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:12.937036 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 674/1329] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:08:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:08:12.942310 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84b3545-94fc-4a8b-ad2b-aadca77db908 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:12.942760 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84b3545-94fc-4a8b-ad2b-aadca77db908 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:12.942980 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84b3545-94fc-4a8b-ad2b-aadca77db908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:13.023498 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-c84b3545-94fc-4a8b-ad2b-aadca77db908 None None] MFA Rules not processed for user `6962122ffdbe4d6288f017e98d1e8ecb`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:08:13.024985 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-c84b3545-94fc-4a8b-ad2b-aadca77db908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:13.096904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:13.131188 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 656/1330] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:08:12 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:08:13.418549 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6977e88-903f-492b-b2ad-7b1dd1aa0250 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:08:13.420194 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f6977e88-903f-492b-b2ad-7b1dd1aa0250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:13.531110 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6977e88-903f-492b-b2ad-7b1dd1aa0250 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:08:13.557151 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:13.614612 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc058d44-3176-4e25-a9ad-ed80a5e39430 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:08:13.615216 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc058d44-3176-4e25-a9ad-ed80a5e39430 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:13.615503 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc058d44-3176-4e25-a9ad-ed80a5e39430 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:13.615768 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc058d44-3176-4e25-a9ad-ed80a5e39430 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:13.617419 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-dc058d44-3176-4e25-a9ad-ed80a5e39430 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:13.735518 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc058d44-3176-4e25-a9ad-ed80a5e39430 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:08:13.737033 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc058d44-3176-4e25-a9ad-ed80a5e39430 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:08:13.756210 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 675/1331] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:08:13 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:08:16.078808 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835b1133-c27e-439c-bb43-5b56f5c0079f None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:16.079363 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835b1133-c27e-439c-bb43-5b56f5c0079f None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:16.079363 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835b1133-c27e-439c-bb43-5b56f5c0079f None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:16.080696 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 657/1332] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:08:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:08:16.086055 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dff656-c906-455d-a124-3d10f40e82f8 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:16.086512 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dff656-c906-455d-a124-3d10f40e82f8 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:16.086610 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dff656-c906-455d-a124-3d10f40e82f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:16.151470 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:16.154594 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-58dff656-c906-455d-a124-3d10f40e82f8 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:08:16.156255 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-58dff656-c906-455d-a124-3d10f40e82f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:16.262093 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 676/1333] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:08:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:08:16.540968 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835b1133-c27e-439c-bb43-5b56f5c0079f None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:08:16.543136 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-835b1133-c27e-439c-bb43-5b56f5c0079f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:16.656713 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835b1133-c27e-439c-bb43-5b56f5c0079f None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:08:16.736983 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb60600-2b6e-46ca-8397-a03df52fb2d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:08:16.737587 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb60600-2b6e-46ca-8397-a03df52fb2d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:16.737859 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb60600-2b6e-46ca-8397-a03df52fb2d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:16.738125 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb60600-2b6e-46ca-8397-a03df52fb2d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:16.739722 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-4cb60600-2b6e-46ca-8397-a03df52fb2d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:16.766807 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:16.854968 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb60600-2b6e-46ca-8397-a03df52fb2d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:08:16.856376 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb60600-2b6e-46ca-8397-a03df52fb2d8 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:08:16.874455 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 658/1334] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:08:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:08:18.823306 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d2479d-75f7-41d0-9549-fe7f772a4cb8 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:18.824194 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d2479d-75f7-41d0-9549-fe7f772a4cb8 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:18.824194 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d2479d-75f7-41d0-9549-fe7f772a4cb8 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:18.825536 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 677/1335] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:08:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:08:18.831360 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3074b01-1eef-47d4-b59c-2091b989b941 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:18.832107 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3074b01-1eef-47d4-b59c-2091b989b941 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:18.832216 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3074b01-1eef-47d4-b59c-2091b989b941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:18.909081 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-a3074b01-1eef-47d4-b59c-2091b989b941 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:08:18.910283 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a3074b01-1eef-47d4-b59c-2091b989b941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:18.993475 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:19.015898 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 659/1336] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:08:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:08:19.296091 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d2479d-75f7-41d0-9549-fe7f772a4cb8 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:08:19.297686 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b6d2479d-75f7-41d0-9549-fe7f772a4cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:19.412362 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d2479d-75f7-41d0-9549-fe7f772a4cb8 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:08:19.440557 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:19.493326 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7037978-f710-4d1e-bdf7-a5ecc5db80fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:08:19.493933 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7037978-f710-4d1e-bdf7-a5ecc5db80fc service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:19.494202 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7037978-f710-4d1e-bdf7-a5ecc5db80fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:19.494463 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7037978-f710-4d1e-bdf7-a5ecc5db80fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:19.496075 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f7037978-f710-4d1e-bdf7-a5ecc5db80fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:19.617398 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7037978-f710-4d1e-bdf7-a5ecc5db80fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:08:19.618750 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7037978-f710-4d1e-bdf7-a5ecc5db80fc service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:08:19.637639 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 678/1337] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:08:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:08:22.036234 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf64922-9717-438b-a9c5-ea9eb9ccbf53 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:22.037110 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf64922-9717-438b-a9c5-ea9eb9ccbf53 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:22.037110 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf64922-9717-438b-a9c5-ea9eb9ccbf53 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:22.039342 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 660/1338] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:08:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:08:22.044931 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a8368e-1d36-4bb8-a1e3-42132225b25c None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:22.045361 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a8368e-1d36-4bb8-a1e3-42132225b25c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:22.045471 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a8368e-1d36-4bb8-a1e3-42132225b25c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:22.115924 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-15a8368e-1d36-4bb8-a1e3-42132225b25c None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:08:22.117160 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-15a8368e-1d36-4bb8-a1e3-42132225b25c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:22.125550 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:22.233000 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 679/1339] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:08:22 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:08:22.515545 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf64922-9717-438b-a9c5-ea9eb9ccbf53 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:08:22.517847 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ddf64922-9717-438b-a9c5-ea9eb9ccbf53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:22.636995 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf64922-9717-438b-a9c5-ea9eb9ccbf53 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:08:22.712597 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5a75ba-cfd2-46ed-83b1-93a740ccf6e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:08:22.713221 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5a75ba-cfd2-46ed-83b1-93a740ccf6e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:22.713470 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5a75ba-cfd2-46ed-83b1-93a740ccf6e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:22.713784 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5a75ba-cfd2-46ed-83b1-93a740ccf6e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:22.715405 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-bd5a75ba-cfd2-46ed-83b1-93a740ccf6e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:22.751556 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:22.830934 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5a75ba-cfd2-46ed-83b1-93a740ccf6e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:08:22.832390 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5a75ba-cfd2-46ed-83b1-93a740ccf6e0 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:08:22.850918 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 661/1340] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:08:22 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:08:26.702345 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e2987-3bc2-425c-8598-3e3c3d104f1f None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:26.702897 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e2987-3bc2-425c-8598-3e3c3d104f1f None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:26.702897 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e2987-3bc2-425c-8598-3e3c3d104f1f None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:26.704133 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 680/1341] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:08:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:08:26.709489 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35b5e4c-ad5c-406d-846e-da5f6864cc17 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:26.709915 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35b5e4c-ad5c-406d-846e-da5f6864cc17 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:26.710013 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35b5e4c-ad5c-406d-846e-da5f6864cc17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:26.781227 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-a35b5e4c-ad5c-406d-846e-da5f6864cc17 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:08:26.782434 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a35b5e4c-ad5c-406d-846e-da5f6864cc17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:26.870613 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:26.883081 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 662/1342] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:08:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:08:27.164264 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1e2987-3bc2-425c-8598-3e3c3d104f1f None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:08:27.165917 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8c1e2987-3bc2-425c-8598-3e3c3d104f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:27.273276 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1e2987-3bc2-425c-8598-3e3c3d104f1f None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:08:27.312821 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:27.351535 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9146082c-9079-402b-bc59-3c2a97914384 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:08:27.352154 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9146082c-9079-402b-bc59-3c2a97914384 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:27.352427 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9146082c-9079-402b-bc59-3c2a97914384 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:27.352692 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9146082c-9079-402b-bc59-3c2a97914384 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:27.354354 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-9146082c-9079-402b-bc59-3c2a97914384 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:27.468145 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9146082c-9079-402b-bc59-3c2a97914384 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:08:27.469524 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9146082c-9079-402b-bc59-3c2a97914384 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:08:27.489187 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 681/1343] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:08:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:08:29.941558 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66e814d-8bdb-4bbf-a96f-d69b6292b5d3 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:29.942131 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66e814d-8bdb-4bbf-a96f-d69b6292b5d3 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:29.942131 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66e814d-8bdb-4bbf-a96f-d69b6292b5d3 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:29.943484 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 663/1344] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:08:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:08:29.949922 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a390ea07-4b9e-47c8-880e-0b8e205c7776 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:29.950452 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a390ea07-4b9e-47c8-880e-0b8e205c7776 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:29.950452 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a390ea07-4b9e-47c8-880e-0b8e205c7776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:30.025164 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-a390ea07-4b9e-47c8-880e-0b8e205c7776 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:08:30.026422 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a390ea07-4b9e-47c8-880e-0b8e205c7776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:30.047530 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:30.136706 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 682/1345] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:08:29 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:08:30.422134 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66e814d-8bdb-4bbf-a96f-d69b6292b5d3 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:08:30.423679 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a66e814d-8bdb-4bbf-a96f-d69b6292b5d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:30.531793 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66e814d-8bdb-4bbf-a96f-d69b6292b5d3 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:08:30.607062 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6b332e-28f6-4722-92cf-767c4360b0aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:08:30.607668 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b332e-28f6-4722-92cf-767c4360b0aa service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:30.607945 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b332e-28f6-4722-92cf-767c4360b0aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:30.608216 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b332e-28f6-4722-92cf-767c4360b0aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:30.609841 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-9a6b332e-28f6-4722-92cf-767c4360b0aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:30.655023 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:30.730898 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6b332e-28f6-4722-92cf-767c4360b0aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:08:30.732416 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6b332e-28f6-4722-92cf-767c4360b0aa service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:08:30.754810 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 664/1346] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:08:30 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:08:34.712330 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b18e11-377c-45d1-9d11-e1cb0883286c None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:34.712888 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b18e11-377c-45d1-9d11-e1cb0883286c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:34.713000 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b18e11-377c-45d1-9d11-e1cb0883286c None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:34.714382 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 683/1347] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:08:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:08:34.720074 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486375b9-0c1d-48e5-b3f2-245cf4f8d985 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:34.720558 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486375b9-0c1d-48e5-b3f2-245cf4f8d985 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:34.720668 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486375b9-0c1d-48e5-b3f2-245cf4f8d985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:34.810220 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-486375b9-0c1d-48e5-b3f2-245cf4f8d985 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:08:34.811431 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-486375b9-0c1d-48e5-b3f2-245cf4f8d985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:34.919899 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:34.924028 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 665/1348] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:08:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:08:35.201619 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b18e11-377c-45d1-9d11-e1cb0883286c None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:08:35.203172 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-94b18e11-377c-45d1-9d11-e1cb0883286c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:35.316593 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b18e11-377c-45d1-9d11-e1cb0883286c None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:08:35.366940 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:35.396780 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5c4eaa-085b-4189-99d9-4fd3788941c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:08:35.397422 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5c4eaa-085b-4189-99d9-4fd3788941c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:35.397697 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5c4eaa-085b-4189-99d9-4fd3788941c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:35.397962 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5c4eaa-085b-4189-99d9-4fd3788941c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:35.399732 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7e5c4eaa-085b-4189-99d9-4fd3788941c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:35.520657 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5c4eaa-085b-4189-99d9-4fd3788941c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:08:35.522154 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5c4eaa-085b-4189-99d9-4fd3788941c7 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:08:35.540635 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 684/1349] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:08:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:08:40.460208 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd31d0b-0df6-4a8e-a91b-0c70a679e97d None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:40.460727 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd31d0b-0df6-4a8e-a91b-0c70a679e97d None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:40.460727 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd31d0b-0df6-4a8e-a91b-0c70a679e97d None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:40.462122 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 666/1350] 10.222.0.43 () {54 vars in 921 bytes} [Tue Sep 6 16:08:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:08:40.467574 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f7cfb2-3cae-41f7-9fcf-4798c5466533 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:40.468012 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f7cfb2-3cae-41f7-9fcf-4798c5466533 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:40.468107 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f7cfb2-3cae-41f7-9fcf-4798c5466533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:40.561793 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-37f7cfb2-3cae-41f7-9fcf-4798c5466533 None None] MFA Rules not processed for user `71c2912a44f04bd983b4d4112dbee734`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:08:40.563009 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-37f7cfb2-3cae-41f7-9fcf-4798c5466533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:40.591792 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:40.736582 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 685/1351] 10.222.0.43 () {58 vars in 1018 bytes} [Tue Sep 6 16:08:40 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:08:44.753009 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318fe6a7-2902-4a26-aafd-aa506effc95f None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:44.753877 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318fe6a7-2902-4a26-aafd-aa506effc95f None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:44.754136 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318fe6a7-2902-4a26-aafd-aa506effc95f None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:44.755670 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 667/1352] 10.222.0.43 () {54 vars in 921 bytes} [Tue Sep 6 16:08:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:08:44.760855 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0491fb52-6baa-4f42-bf0a-dd98e3a5456f None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:44.761302 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0491fb52-6baa-4f42-bf0a-dd98e3a5456f None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:44.761483 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0491fb52-6baa-4f42-bf0a-dd98e3a5456f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:44.839780 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-0491fb52-6baa-4f42-bf0a-dd98e3a5456f None None] MFA Rules not processed for user `71c2912a44f04bd983b4d4112dbee734`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:08:44.841058 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0491fb52-6baa-4f42-bf0a-dd98e3a5456f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:44.922023 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:44.964164 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 686/1353] 10.222.0.43 () {58 vars in 1018 bytes} [Tue Sep 6 16:08:44 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:08:46.431171 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3d8fac-6624-4682-a498-98cfd6f358c9 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:46.431705 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3d8fac-6624-4682-a498-98cfd6f358c9 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:46.431705 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3d8fac-6624-4682-a498-98cfd6f358c9 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:46.433179 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 668/1354] 10.222.0.43 () {54 vars in 921 bytes} [Tue Sep 6 16:08:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:08:46.438626 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678a37d4-267f-4c61-a5fc-7afc003a9f8b None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:46.439117 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678a37d4-267f-4c61-a5fc-7afc003a9f8b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:46.439117 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678a37d4-267f-4c61-a5fc-7afc003a9f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:46.520271 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-678a37d4-267f-4c61-a5fc-7afc003a9f8b None None] MFA Rules not processed for user `71c2912a44f04bd983b4d4112dbee734`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:08:46.521490 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-678a37d4-267f-4c61-a5fc-7afc003a9f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:46.632901 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 687/1355] 10.222.0.43 () {58 vars in 1018 bytes} [Tue Sep 6 16:08:46 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:08:49.982140 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36f35e4-c47a-4f3e-81bf-72539f5ba718 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:49.982619 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36f35e4-c47a-4f3e-81bf-72539f5ba718 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:49.982619 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36f35e4-c47a-4f3e-81bf-72539f5ba718 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:49.983997 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 669/1356] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:08:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:08:49.989502 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426c48cd-3361-43be-94f7-ca62d433b861 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:49.989954 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426c48cd-3361-43be-94f7-ca62d433b861 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:49.990062 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426c48cd-3361-43be-94f7-ca62d433b861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:50.034853 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:50.076958 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-426c48cd-3361-43be-94f7-ca62d433b861 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:08:50.078174 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-426c48cd-3361-43be-94f7-ca62d433b861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:50.186717 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 688/1357] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:08:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:08:50.491829 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d79ee7-a2ac-4293-a4dd-bf791aebf8cb None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:50.492496 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d79ee7-a2ac-4293-a4dd-bf791aebf8cb None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:50.493426 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d79ee7-a2ac-4293-a4dd-bf791aebf8cb None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:50.496326 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 670/1358] 10.222.0.43 () {54 vars in 962 bytes} [Tue Sep 6 16:08:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:08:50.503632 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7731600a-410b-4cb6-9986-0b910a37cd43 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:50.504040 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7731600a-410b-4cb6-9986-0b910a37cd43 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:50.504274 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7731600a-410b-4cb6-9986-0b910a37cd43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:50.587047 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-7731600a-410b-4cb6-9986-0b910a37cd43 None None] MFA Rules not processed for user `63de41f4348243998029f7d07aa5dc40`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:08:50.588483 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7731600a-410b-4cb6-9986-0b910a37cd43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:50.638025 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:50.720124 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 689/1359] 10.222.0.43 () {58 vars in 1059 bytes} [Tue Sep 6 16:08:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:08:50.721309 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d79ee7-a2ac-4293-a4dd-bf791aebf8cb None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:08:50.723114 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a3d79ee7-a2ac-4293-a4dd-bf791aebf8cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:50.878350 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d79ee7-a2ac-4293-a4dd-bf791aebf8cb None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:08:50.977109 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:51.007151 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a88b36-9c87-42ef-8060-a25914e23907 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63de41f4348243998029f7d07aa5dc40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:08:51.007800 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a88b36-9c87-42ef-8060-a25914e23907 service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:51.008129 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a88b36-9c87-42ef-8060-a25914e23907 service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:51.008497 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a88b36-9c87-42ef-8060-a25914e23907 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:51.010378 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-22a88b36-9c87-42ef-8060-a25914e23907 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:51.104490 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a88b36-9c87-42ef-8060-a25914e23907 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:08:51.106452 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a88b36-9c87-42ef-8060-a25914e23907 service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:08:51.132807 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 671/1360] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:08:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:08:53.237297 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd36d7c-a14d-4bdd-a0f7-9b53dc174395 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:53.237847 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd36d7c-a14d-4bdd-a0f7-9b53dc174395 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:53.237847 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd36d7c-a14d-4bdd-a0f7-9b53dc174395 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:53.240041 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 690/1361] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:08:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:08:53.247914 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e144285-c46e-453c-8217-1fb46a1d1f2e None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:53.248953 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e144285-c46e-453c-8217-1fb46a1d1f2e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:53.249208 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e144285-c46e-453c-8217-1fb46a1d1f2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:53.325562 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-1e144285-c46e-453c-8217-1fb46a1d1f2e None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:08:53.326778 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1e144285-c46e-453c-8217-1fb46a1d1f2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:53.365866 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:53.449702 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 672/1362] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:08:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:08:53.734300 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd36d7c-a14d-4bdd-a0f7-9b53dc174395 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:08:53.735933 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8dd36d7c-a14d-4bdd-a0f7-9b53dc174395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:53.889027 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd36d7c-a14d-4bdd-a0f7-9b53dc174395 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:08:53.898673 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:54.011358 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a5ce23-7ad0-479e-9861-15386db361b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63de41f4348243998029f7d07aa5dc40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:08:54.012076 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a5ce23-7ad0-479e-9861-15386db361b2 service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:54.012424 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a5ce23-7ad0-479e-9861-15386db361b2 service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:54.012896 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a5ce23-7ad0-479e-9861-15386db361b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:54.014635 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-92a5ce23-7ad0-479e-9861-15386db361b2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:54.114769 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a5ce23-7ad0-479e-9861-15386db361b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:08:54.117005 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a5ce23-7ad0-479e-9861-15386db361b2 service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:08:54.137764 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 691/1363] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:08:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:08:56.335906 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d384bf5-e741-4870-ae25-62e19263b836 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:56.336803 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d384bf5-e741-4870-ae25-62e19263b836 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:56.337319 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d384bf5-e741-4870-ae25-62e19263b836 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:56.338860 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 673/1364] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:08:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:08:56.344495 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88f0712-483f-4c36-ab6d-11d6f4eac447 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:56.345664 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88f0712-483f-4c36-ab6d-11d6f4eac447 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:56.346001 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88f0712-483f-4c36-ab6d-11d6f4eac447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:56.363979 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:56.419095 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-b88f0712-483f-4c36-ab6d-11d6f4eac447 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:08:56.420295 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b88f0712-483f-4c36-ab6d-11d6f4eac447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:56.520004 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 692/1365] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:08:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:08:56.802857 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fba015-e770-4248-ad32-b567dda756c3 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:56.803689 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fba015-e770-4248-ad32-b567dda756c3 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:56.804101 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fba015-e770-4248-ad32-b567dda756c3 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:56.806361 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 674/1366] 10.222.0.43 () {54 vars in 962 bytes} [Tue Sep 6 16:08:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:08:56.812449 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adf88c4-a268-4889-9ca3-2b04fa46e7d6 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:56.812994 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adf88c4-a268-4889-9ca3-2b04fa46e7d6 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:56.813380 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adf88c4-a268-4889-9ca3-2b04fa46e7d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:56.897514 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-8adf88c4-a268-4889-9ca3-2b04fa46e7d6 None None] MFA Rules not processed for user `63de41f4348243998029f7d07aa5dc40`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:08:56.898902 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8adf88c4-a268-4889-9ca3-2b04fa46e7d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:56.936977 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:57.025654 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 693/1367] 10.222.0.43 () {58 vars in 1059 bytes} [Tue Sep 6 16:08:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:08:57.034031 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fba015-e770-4248-ad32-b567dda756c3 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:08:57.035630 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-46fba015-e770-4248-ad32-b567dda756c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:57.191203 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fba015-e770-4248-ad32-b567dda756c3 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:08:57.212205 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:57.319871 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07f21ff-5771-468d-aa0c-baf4b049f430 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63de41f4348243998029f7d07aa5dc40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:08:57.320501 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07f21ff-5771-468d-aa0c-baf4b049f430 service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:57.320787 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07f21ff-5771-468d-aa0c-baf4b049f430 service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:57.321041 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07f21ff-5771-468d-aa0c-baf4b049f430 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:57.323140 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f07f21ff-5771-468d-aa0c-baf4b049f430 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:57.415974 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07f21ff-5771-468d-aa0c-baf4b049f430 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:08:57.417665 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07f21ff-5771-468d-aa0c-baf4b049f430 service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:08:57.439095 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 675/1368] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:08:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:08:58.058308 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c4337b-2612-4d52-816f-dc04dd391876 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:58.058864 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c4337b-2612-4d52-816f-dc04dd391876 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:58.058864 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c4337b-2612-4d52-816f-dc04dd391876 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:58.060425 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 694/1369] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:08:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:08:58.065598 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea89d28-999c-49fc-8c56-a822ee048f05 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:58.065886 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea89d28-999c-49fc-8c56-a822ee048f05 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:58.066174 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea89d28-999c-49fc-8c56-a822ee048f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:58.100065 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:58.137873 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-7ea89d28-999c-49fc-8c56-a822ee048f05 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:08:58.139167 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-7ea89d28-999c-49fc-8c56-a822ee048f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:58.240580 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 676/1370] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:08:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:08:58.829442 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d401ebb-f830-4033-a723-fcae940ac815 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:58.829442 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d401ebb-f830-4033-a723-fcae940ac815 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:58.829442 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d401ebb-f830-4033-a723-fcae940ac815 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:58.829442 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 695/1371] 10.222.0.43 () {54 vars in 964 bytes} [Tue Sep 6 16:08:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:08:58.833795 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1f19c3-ca58-4e7a-b600-234275bc0059 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:58.833795 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1f19c3-ca58-4e7a-b600-234275bc0059 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:58.833795 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1f19c3-ca58-4e7a-b600-234275bc0059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:58.911979 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-bf1f19c3-ca58-4e7a-b600-234275bc0059 None None] MFA Rules not processed for user `d1b05748e4284153b0728164cd508dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:08:58.913337 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-bf1f19c3-ca58-4e7a-b600-234275bc0059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:58.962801 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:58.997529 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 677/1372] 10.222.0.43 () {58 vars in 1061 bytes} [Tue Sep 6 16:08:58 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:08:59.005611 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d401ebb-f830-4033-a723-fcae940ac815 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:08:59.007276 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-6d401ebb-f830-4033-a723-fcae940ac815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:59.120405 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d401ebb-f830-4033-a723-fcae940ac815 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:08:59.168064 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:59.210288 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdeb08f-244e-489c-b201-cbebd3f0cc1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:08:59.211047 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdeb08f-244e-489c-b201-cbebd3f0cc1a service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:59.211460 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdeb08f-244e-489c-b201-cbebd3f0cc1a service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:59.211966 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdeb08f-244e-489c-b201-cbebd3f0cc1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:59.213958 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-4bdeb08f-244e-489c-b201-cbebd3f0cc1a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:59.358399 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdeb08f-244e-489c-b201-cbebd3f0cc1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:08:59.359751 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdeb08f-244e-489c-b201-cbebd3f0cc1a service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:08:59.386681 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 696/1373] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:08:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:08:59.387784 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6804d6-59dc-443f-877d-432638eab39e None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:59.387975 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6804d6-59dc-443f-877d-432638eab39e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:59.388230 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6804d6-59dc-443f-877d-432638eab39e None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:59.389747 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 678/1374] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:08:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:08:59.394782 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a319a2f9-2663-4f2f-84cf-6becdb97ec8c None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:59.395066 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a319a2f9-2663-4f2f-84cf-6becdb97ec8c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:59.395376 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a319a2f9-2663-4f2f-84cf-6becdb97ec8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:59.482688 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-a319a2f9-2663-4f2f-84cf-6becdb97ec8c None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:08:59.484465 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a319a2f9-2663-4f2f-84cf-6becdb97ec8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:59.510402 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:59.586554 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 697/1375] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:08:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:08:59.872460 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6804d6-59dc-443f-877d-432638eab39e None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:08:59.874025 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ad6804d6-59dc-443f-877d-432638eab39e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:00.028922 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6804d6-59dc-443f-877d-432638eab39e None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:09:00.086246 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:09:00.150568 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8291462-a214-4907-9088-4919dcb2cecd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63de41f4348243998029f7d07aa5dc40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:09:00.151102 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8291462-a214-4907-9088-4919dcb2cecd service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:00.151382 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8291462-a214-4907-9088-4919dcb2cecd service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:00.151650 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8291462-a214-4907-9088-4919dcb2cecd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:00.153302 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e8291462-a214-4907-9088-4919dcb2cecd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:00.247455 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8291462-a214-4907-9088-4919dcb2cecd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:09:00.249415 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8291462-a214-4907-9088-4919dcb2cecd service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:09:00.267987 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 679/1376] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:08:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:09:02.216156 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dff16e8-7b81-49c7-be28-354db67889d6 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:02.216683 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dff16e8-7b81-49c7-be28-354db67889d6 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:02.216683 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dff16e8-7b81-49c7-be28-354db67889d6 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:02.218073 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 698/1377] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:09:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:09:02.223735 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f8abda-d493-4b68-b47e-41f1b5c00e19 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:02.224151 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f8abda-d493-4b68-b47e-41f1b5c00e19 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:02.224392 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f8abda-d493-4b68-b47e-41f1b5c00e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:02.299134 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-a7f8abda-d493-4b68-b47e-41f1b5c00e19 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:09:02.300398 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a7f8abda-d493-4b68-b47e-41f1b5c00e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:02.313779 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:09:02.420185 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 680/1378] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:09:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:09:02.726006 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dff16e8-7b81-49c7-be28-354db67889d6 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:09:02.727546 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-1dff16e8-7b81-49c7-be28-354db67889d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:03.036181 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dff16e8-7b81-49c7-be28-354db67889d6 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:09:03.039921 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:09:03.161719 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b40d87-158f-484f-8891-9188eafccc4e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63de41f4348243998029f7d07aa5dc40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:09:03.162640 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b40d87-158f-484f-8891-9188eafccc4e service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:03.162916 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b40d87-158f-484f-8891-9188eafccc4e service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:03.163179 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b40d87-158f-484f-8891-9188eafccc4e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:03.164841 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e4b40d87-158f-484f-8891-9188eafccc4e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:03.267385 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b40d87-158f-484f-8891-9188eafccc4e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:09:03.269224 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b40d87-158f-484f-8891-9188eafccc4e service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:09:03.293010 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 699/1379] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:09:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 567 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:09:05.230683 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8129a1d-c53d-42d0-acde-d1e1a1f31149 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:05.231319 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8129a1d-c53d-42d0-acde-d1e1a1f31149 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:05.231319 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8129a1d-c53d-42d0-acde-d1e1a1f31149 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:05.232558 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 681/1380] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:09:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:09:05.237982 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a06002-8134-4631-82ec-a678245edc1b None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:05.238440 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a06002-8134-4631-82ec-a678245edc1b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:05.238547 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a06002-8134-4631-82ec-a678245edc1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:05.256238 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:09:05.319760 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-50a06002-8134-4631-82ec-a678245edc1b None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:09:05.320927 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-50a06002-8134-4631-82ec-a678245edc1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:05.422898 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 700/1381] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:09:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:09:05.709892 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8129a1d-c53d-42d0-acde-d1e1a1f31149 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:09:05.711421 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d8129a1d-c53d-42d0-acde-d1e1a1f31149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:05.865258 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:09:05.878744 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8129a1d-c53d-42d0-acde-d1e1a1f31149 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:09:05.995849 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8192efd-77de-4120-a2cd-8fad3cfc589c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63de41f4348243998029f7d07aa5dc40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:09:05.996645 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8192efd-77de-4120-a2cd-8fad3cfc589c service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:05.997042 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8192efd-77de-4120-a2cd-8fad3cfc589c service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:05.997401 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8192efd-77de-4120-a2cd-8fad3cfc589c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:05.999129 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f8192efd-77de-4120-a2cd-8fad3cfc589c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:06.096426 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8192efd-77de-4120-a2cd-8fad3cfc589c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:09:06.098315 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8192efd-77de-4120-a2cd-8fad3cfc589c service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:09:06.110305 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:09:06.119151 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 682/1382] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:09:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:09:08.082143 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d5f9f6-6346-4f7d-8b16-accff1d2ef97 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:08.082916 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d5f9f6-6346-4f7d-8b16-accff1d2ef97 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:08.082916 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d5f9f6-6346-4f7d-8b16-accff1d2ef97 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:08.084013 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 701/1383] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:09:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:09:08.090391 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a3bdb6-b482-4243-af2d-f2a007c20232 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:08.090902 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a3bdb6-b482-4243-af2d-f2a007c20232 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:08.091223 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a3bdb6-b482-4243-af2d-f2a007c20232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:08.169431 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-71a3bdb6-b482-4243-af2d-f2a007c20232 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:09:08.170743 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-71a3bdb6-b482-4243-af2d-f2a007c20232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:08.280899 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 683/1384] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:09:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:09:08.563037 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d5f9f6-6346-4f7d-8b16-accff1d2ef97 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:09:08.564690 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d0d5f9f6-6346-4f7d-8b16-accff1d2ef97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:08.660193 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:09:08.723746 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d5f9f6-6346-4f7d-8b16-accff1d2ef97 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:09:08.855645 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbac574-1424-4a8e-9c14-5caabf9bc5d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63de41f4348243998029f7d07aa5dc40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:09:08.856175 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbac574-1424-4a8e-9c14-5caabf9bc5d9 service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:08.856438 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbac574-1424-4a8e-9c14-5caabf9bc5d9 service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:08.856702 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbac574-1424-4a8e-9c14-5caabf9bc5d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:08.858363 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0fbac574-1424-4a8e-9c14-5caabf9bc5d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:08.910748 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:09:08.956348 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbac574-1424-4a8e-9c14-5caabf9bc5d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:09:08.958124 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbac574-1424-4a8e-9c14-5caabf9bc5d9 service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:09:08.977516 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 702/1385] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:09:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:09:10.896483 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cd05e2-d8b8-43de-b05f-52e9cd0722d2 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:10.897171 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cd05e2-d8b8-43de-b05f-52e9cd0722d2 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:10.897282 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cd05e2-d8b8-43de-b05f-52e9cd0722d2 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:10.899237 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 684/1386] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:09:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:09:10.906151 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c637aa-7aa5-4af0-a095-ab948464b09f None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:10.906588 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c637aa-7aa5-4af0-a095-ab948464b09f None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:10.906741 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c637aa-7aa5-4af0-a095-ab948464b09f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:10.987641 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-81c637aa-7aa5-4af0-a095-ab948464b09f None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:09:10.989431 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-81c637aa-7aa5-4af0-a095-ab948464b09f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:11.100160 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 703/1387] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:09:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:09:11.387340 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cd05e2-d8b8-43de-b05f-52e9cd0722d2 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:09:11.389659 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-44cd05e2-d8b8-43de-b05f-52e9cd0722d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:11.486584 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:09:11.577584 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cd05e2-d8b8-43de-b05f-52e9cd0722d2 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:09:11.725830 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1fc73c-e0d9-47d7-b7d7-12583d5ae470 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63de41f4348243998029f7d07aa5dc40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:09:11.726537 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1fc73c-e0d9-47d7-b7d7-12583d5ae470 service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:11.726983 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1fc73c-e0d9-47d7-b7d7-12583d5ae470 service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:11.727335 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1fc73c-e0d9-47d7-b7d7-12583d5ae470 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:11.729541 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ec1fc73c-e0d9-47d7-b7d7-12583d5ae470 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:11.781759 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:09:11.839187 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1fc73c-e0d9-47d7-b7d7-12583d5ae470 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:09:11.840901 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1fc73c-e0d9-47d7-b7d7-12583d5ae470 service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:09:11.860056 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 685/1388] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:09:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:09:13.851261 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f5c35b-f2a9-4ea6-b90e-4781920f2223 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:13.851879 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f5c35b-f2a9-4ea6-b90e-4781920f2223 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:13.851879 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f5c35b-f2a9-4ea6-b90e-4781920f2223 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:13.853249 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 704/1389] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:09:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:09:13.858711 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e986340b-0270-4041-a457-44bb91c038fe None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:13.859151 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e986340b-0270-4041-a457-44bb91c038fe None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:13.859249 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e986340b-0270-4041-a457-44bb91c038fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:13.929715 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-e986340b-0270-4041-a457-44bb91c038fe None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:09:13.931044 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e986340b-0270-4041-a457-44bb91c038fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:14.031817 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 686/1390] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:09:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:09:14.206250 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7efc68e-4f6a-427a-8bd0-b3b49960bd33 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:14.206453 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7efc68e-4f6a-427a-8bd0-b3b49960bd33 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:14.206711 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7efc68e-4f6a-427a-8bd0-b3b49960bd33 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:14.208158 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 705/1391] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:09:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:09:14.213610 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a896751-9971-4dc1-8b43-f87f30318b2b None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:14.213997 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a896751-9971-4dc1-8b43-f87f30318b2b None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:14.214270 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a896751-9971-4dc1-8b43-f87f30318b2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:14.230659 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:09:14.284955 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-6a896751-9971-4dc1-8b43-f87f30318b2b None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:09:14.286182 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6a896751-9971-4dc1-8b43-f87f30318b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:14.315149 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7efc68e-4f6a-427a-8bd0-b3b49960bd33 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:09:14.316706 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b7efc68e-4f6a-427a-8bd0-b3b49960bd33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:14.349156 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:09:14.386047 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 687/1392] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:09:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:09:14.463898 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7efc68e-4f6a-427a-8bd0-b3b49960bd33 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:09:14.562684 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:09:14.575676 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bdc6c4-c7af-4bfa-b205-758216a78947 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63de41f4348243998029f7d07aa5dc40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:09:14.576285 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bdc6c4-c7af-4bfa-b205-758216a78947 service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:14.576554 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bdc6c4-c7af-4bfa-b205-758216a78947 service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:14.576851 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bdc6c4-c7af-4bfa-b205-758216a78947 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:14.578433 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d3bdc6c4-c7af-4bfa-b205-758216a78947 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:14.667590 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bdc6c4-c7af-4bfa-b205-758216a78947 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:09:14.669231 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bdc6c4-c7af-4bfa-b205-758216a78947 service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:09:14.680777 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd775d5-8e0c-4fa3-8048-97423406a210 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:14.681221 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd775d5-8e0c-4fa3-8048-97423406a210 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:14.681502 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd775d5-8e0c-4fa3-8048-97423406a210 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:14.682901 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 688/1393] 10.222.0.43 () {54 vars in 961 bytes} [Tue Sep 6 16:09:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 16:09:14.687966 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f3ff7d-e7e9-4ce8-8e1c-cc4446088aab None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:14.688276 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f3ff7d-e7e9-4ce8-8e1c-cc4446088aab None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:14.688570 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f3ff7d-e7e9-4ce8-8e1c-cc4446088aab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:14.688797 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 706/1394] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:09:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:09:14.770248 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-62f3ff7d-e7e9-4ce8-8e1c-cc4446088aab None None] MFA Rules not processed for user `2c163b39e61648c385f065ce8091c3ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:09:14.771523 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-62f3ff7d-e7e9-4ce8-8e1c-cc4446088aab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:14.812218 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:09:14.860232 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 689/1395] 10.222.0.43 () {58 vars in 1058 bytes} [Tue Sep 6 16:09:14 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:09:14.868612 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bdc6c4-c7af-4bfa-b205-758216a78947 service nova] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:09:14.869884 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d3bdc6c4-c7af-4bfa-b205-758216a78947 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:14.952826 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bdc6c4-c7af-4bfa-b205-758216a78947 service nova] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:09:14.994559 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:09:15.028247 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc27426-a944-444d-97a3-4cda84d391f2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c163b39e61648c385f065ce8091c3ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:09:15.028892 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc27426-a944-444d-97a3-4cda84d391f2 service glance] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:15.029169 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc27426-a944-444d-97a3-4cda84d391f2 service glance] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:15.029438 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc27426-a944-444d-97a3-4cda84d391f2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:15.031037 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-cbc27426-a944-444d-97a3-4cda84d391f2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:15.141617 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc27426-a944-444d-97a3-4cda84d391f2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:09:15.143060 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc27426-a944-444d-97a3-4cda84d391f2 service glance] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:09:15.161923 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 707/1396] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:09:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:09:15.917051 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783f77b0-2711-494f-823f-784bf96f1450 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:15.917531 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783f77b0-2711-494f-823f-784bf96f1450 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:15.917632 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783f77b0-2711-494f-823f-784bf96f1450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:15.986096 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-783f77b0-2711-494f-823f-784bf96f1450 None None] MFA Rules not processed for user `d865006413b24ec3922619311f81f353`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:09:15.987296 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-783f77b0-2711-494f-823f-784bf96f1450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:16.074128 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 690/1397] 10.222.0.43 () {58 vars in 1009 bytes} [Tue Sep 6 16:09:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:09:16.115774 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeaac6a-a9a9-4af0-a114-891809795d5f None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:16.116262 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeaac6a-a9a9-4af0-a114-891809795d5f None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:16.116352 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeaac6a-a9a9-4af0-a114-891809795d5f None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:16.117800 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 708/1398] 10.222.0.43 () {54 vars in 943 bytes} [Tue Sep 6 16:09:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:09:16.124455 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f76f7b7-332d-4c78-a95a-a08deda2ebac None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:16.124713 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f76f7b7-332d-4c78-a95a-a08deda2ebac None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:16.125037 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f76f7b7-332d-4c78-a95a-a08deda2ebac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:16.183668 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:09:16.205631 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-6f76f7b7-332d-4c78-a95a-a08deda2ebac None None] MFA Rules not processed for user `312026c080b64aff9dd483d3591288a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:09:16.206851 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6f76f7b7-332d-4c78-a95a-a08deda2ebac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:16.326871 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 691/1399] 10.222.0.43 () {58 vars in 1040 bytes} [Tue Sep 6 16:09:16 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:09:16.336479 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeaac6a-a9a9-4af0-a114-891809795d5f None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:09:16.338132 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-abeaac6a-a9a9-4af0-a114-891809795d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:16.418933 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:09:16.479442 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeaac6a-a9a9-4af0-a114-891809795d5f None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:09:16.587003 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c1355d-328b-4686-ad79-11570f26513d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '312026c080b64aff9dd483d3591288a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:09:16.587614 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c1355d-328b-4686-ad79-11570f26513d service swift] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:16.587884 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c1355d-328b-4686-ad79-11570f26513d service swift] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:16.588161 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c1355d-328b-4686-ad79-11570f26513d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:16.589794 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-45c1355d-328b-4686-ad79-11570f26513d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:16.632351 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42f8f54-7d03-43ab-b6bb-02ad3b9b2614 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:16.632704 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42f8f54-7d03-43ab-b6bb-02ad3b9b2614 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:16.632920 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42f8f54-7d03-43ab-b6bb-02ad3b9b2614 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:16.634350 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 692/1400] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:09:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 16:09:16.639259 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e324aa0c-da84-49fb-9b8c-ff281b12f500 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:16.639515 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e324aa0c-da84-49fb-9b8c-ff281b12f500 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:16.639787 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e324aa0c-da84-49fb-9b8c-ff281b12f500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:16.653094 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:09:16.679749 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c1355d-328b-4686-ad79-11570f26513d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:09:16.681343 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c1355d-328b-4686-ad79-11570f26513d service swift] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:09:16.689577 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 709/1401] 10.222.0.43 () {60 vars in 1384 bytes} [Tue Sep 6 16:09:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 16:09:16.708441 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-e324aa0c-da84-49fb-9b8c-ff281b12f500 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:09:16.709677 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e324aa0c-da84-49fb-9b8c-ff281b12f500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:16.771001 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:09:16.819158 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 693/1402] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:09:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:09:17.110657 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c1355d-328b-4686-ad79-11570f26513d service swift] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:09:17.111089 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-45c1355d-328b-4686-ad79-11570f26513d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:17.207220 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c1355d-328b-4686-ad79-11570f26513d service swift] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:09:17.323932 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:09:17.336840 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4032e560-b99f-4298-98c9-8d1a9c243e1d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63de41f4348243998029f7d07aa5dc40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:09:17.337892 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4032e560-b99f-4298-98c9-8d1a9c243e1d service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:17.338478 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4032e560-b99f-4298-98c9-8d1a9c243e1d service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:17.338912 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4032e560-b99f-4298-98c9-8d1a9c243e1d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:17.340771 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-4032e560-b99f-4298-98c9-8d1a9c243e1d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:17.444180 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4032e560-b99f-4298-98c9-8d1a9c243e1d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:09:17.446847 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4032e560-b99f-4298-98c9-8d1a9c243e1d service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:09:17.464986 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 710/1403] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:09:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:09:19.456309 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1019e5a0-1af8-473d-8477-5d514a8a67fb None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:19.457137 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1019e5a0-1af8-473d-8477-5d514a8a67fb None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:19.457137 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1019e5a0-1af8-473d-8477-5d514a8a67fb None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:19.458289 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 694/1404] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:09:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:09:19.463770 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f24027-2737-4181-9eb7-92d091a45f2d None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:19.464193 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f24027-2737-4181-9eb7-92d091a45f2d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:19.464371 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f24027-2737-4181-9eb7-92d091a45f2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:19.536038 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-e8f24027-2737-4181-9eb7-92d091a45f2d None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:09:19.537601 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e8f24027-2737-4181-9eb7-92d091a45f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:19.598902 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:09:19.642082 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 711/1405] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:09:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:09:19.926149 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1019e5a0-1af8-473d-8477-5d514a8a67fb None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:09:19.927679 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1019e5a0-1af8-473d-8477-5d514a8a67fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:20.086925 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1019e5a0-1af8-473d-8477-5d514a8a67fb None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:09:20.129090 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:09:20.214884 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb782810-6715-45dc-9acc-6f9214bace5d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63de41f4348243998029f7d07aa5dc40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:09:20.215379 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb782810-6715-45dc-9acc-6f9214bace5d service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:20.215649 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb782810-6715-45dc-9acc-6f9214bace5d service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:20.215913 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb782810-6715-45dc-9acc-6f9214bace5d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:20.217551 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-bb782810-6715-45dc-9acc-6f9214bace5d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:20.323457 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb782810-6715-45dc-9acc-6f9214bace5d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:09:20.325303 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb782810-6715-45dc-9acc-6f9214bace5d service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:09:20.344853 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 695/1406] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:09:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:09:22.301669 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d89f0f-4bdc-48c6-8935-1c204dff6970 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:22.301669 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d89f0f-4bdc-48c6-8935-1c204dff6970 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:22.301669 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d89f0f-4bdc-48c6-8935-1c204dff6970 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:22.307072 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 712/1407] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:09:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:09:22.308907 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afc22d2-5831-4f43-9349-0084c9d84e87 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:22.309707 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afc22d2-5831-4f43-9349-0084c9d84e87 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:22.310142 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afc22d2-5831-4f43-9349-0084c9d84e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:22.390847 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:09:22.398344 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-0afc22d2-5831-4f43-9349-0084c9d84e87 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:09:22.399681 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0afc22d2-5831-4f43-9349-0084c9d84e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:22.514399 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 696/1408] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:09:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:09:22.805911 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d89f0f-4bdc-48c6-8935-1c204dff6970 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:09:22.807438 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-31d89f0f-4bdc-48c6-8935-1c204dff6970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:22.964676 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d89f0f-4bdc-48c6-8935-1c204dff6970 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:09:23.020330 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:09:23.095270 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714c6875-e97c-4efd-b74a-532f22ba8420 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63de41f4348243998029f7d07aa5dc40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:09:23.096048 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c6875-e97c-4efd-b74a-532f22ba8420 service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:23.096327 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c6875-e97c-4efd-b74a-532f22ba8420 service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:23.096620 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c6875-e97c-4efd-b74a-532f22ba8420 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:23.098851 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-714c6875-e97c-4efd-b74a-532f22ba8420 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:23.199240 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714c6875-e97c-4efd-b74a-532f22ba8420 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:09:23.201130 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714c6875-e97c-4efd-b74a-532f22ba8420 service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:09:23.226688 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 713/1409] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:09:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:09:32.511687 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=44 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:09:33.223142 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=77 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:09:38.602542 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d68206-a011-4354-8f25-a96bfb71215f None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:38.603153 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d68206-a011-4354-8f25-a96bfb71215f None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:38.603153 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d68206-a011-4354-8f25-a96bfb71215f None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:38.604780 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 697/1410] 10.222.0.43 () {54 vars in 918 bytes} [Tue Sep 6 16:09:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:09:38.610297 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a573d7ec-672d-49f0-8956-ecddd503139b None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:38.610754 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a573d7ec-672d-49f0-8956-ecddd503139b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:38.610857 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a573d7ec-672d-49f0-8956-ecddd503139b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:38.688790 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-a573d7ec-672d-49f0-8956-ecddd503139b None None] MFA Rules not processed for user `71c2912a44f04bd983b4d4112dbee734`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:09:38.690094 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a573d7ec-672d-49f0-8956-ecddd503139b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:38.807834 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 714/1411] 10.222.0.43 () {58 vars in 1015 bytes} [Tue Sep 6 16:09:38 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:09:38.823045 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5404ef0d-0fa0-4895-89ae-e50bba6b7e13 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:38.823381 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5404ef0d-0fa0-4895-89ae-e50bba6b7e13 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:38.823655 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5404ef0d-0fa0-4895-89ae-e50bba6b7e13 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:38.825062 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 698/1412] 10.222.0.43 () {54 vars in 918 bytes} [Tue Sep 6 16:09:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:09:38.829828 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069557f4-4d9a-40e7-bf52-9e41782fbeff None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:38.830141 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069557f4-4d9a-40e7-bf52-9e41782fbeff None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:38.830418 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069557f4-4d9a-40e7-bf52-9e41782fbeff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:38.905941 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:09:38.910868 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-069557f4-4d9a-40e7-bf52-9e41782fbeff None None] MFA Rules not processed for user `63de41f4348243998029f7d07aa5dc40`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:09:38.913732 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-069557f4-4d9a-40e7-bf52-9e41782fbeff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:39.050325 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 715/1413] 10.222.0.43 () {58 vars in 1015 bytes} [Tue Sep 6 16:09:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:09:41.427303 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007850cf-f1d4-4a7c-849a-ebbd6745a80e None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:41.427879 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007850cf-f1d4-4a7c-849a-ebbd6745a80e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:41.427879 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007850cf-f1d4-4a7c-849a-ebbd6745a80e None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:41.429144 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 699/1414] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 16:09:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:09:41.435495 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d707f711-7a19-4211-a963-53ce8ff9f75a None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:41.436217 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d707f711-7a19-4211-a963-53ce8ff9f75a None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:41.436338 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d707f711-7a19-4211-a963-53ce8ff9f75a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:41.524178 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-d707f711-7a19-4211-a963-53ce8ff9f75a None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:09:41.525863 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d707f711-7a19-4211-a963-53ce8ff9f75a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:41.582732 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:09:41.630262 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 716/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 16:09:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:09:41.924554 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007850cf-f1d4-4a7c-849a-ebbd6745a80e None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:09:41.926180 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-007850cf-f1d4-4a7c-849a-ebbd6745a80e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:42.078910 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007850cf-f1d4-4a7c-849a-ebbd6745a80e None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:09:42.163199 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:09:42.205297 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b574b160-375e-4aab-a994-e36dcc7a4603 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63de41f4348243998029f7d07aa5dc40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:09:42.205909 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b574b160-375e-4aab-a994-e36dcc7a4603 service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:42.206182 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b574b160-375e-4aab-a994-e36dcc7a4603 service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:42.206452 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b574b160-375e-4aab-a994-e36dcc7a4603 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:42.208050 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b574b160-375e-4aab-a994-e36dcc7a4603 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:42.300642 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b574b160-375e-4aab-a994-e36dcc7a4603 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:09:42.302320 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b574b160-375e-4aab-a994-e36dcc7a4603 service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:09:42.321707 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 700/1416] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:09:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:09:43.204823 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8da51a2-84bb-43b1-b927-a5d3275ecd2c None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:43.205435 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8da51a2-84bb-43b1-b927-a5d3275ecd2c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:43.205435 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8da51a2-84bb-43b1-b927-a5d3275ecd2c None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:43.206749 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 717/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 16:09:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 16:09:43.213018 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e6ff56-3ff2-40c5-ac4c-0f5a6df34c21 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:43.214036 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e6ff56-3ff2-40c5-ac4c-0f5a6df34c21 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:43.214036 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e6ff56-3ff2-40c5-ac4c-0f5a6df34c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:43.293713 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-89e6ff56-3ff2-40c5-ac4c-0f5a6df34c21 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:09:43.294922 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-89e6ff56-3ff2-40c5-ac4c-0f5a6df34c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:43.323887 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:09:43.399491 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 701/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 16:09:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:09:43.550561 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8da51a2-84bb-43b1-b927-a5d3275ecd2c None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:09:43.552925 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f8da51a2-84bb-43b1-b927-a5d3275ecd2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:43.718002 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8da51a2-84bb-43b1-b927-a5d3275ecd2c None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:09:43.761293 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:09:43.838573 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0296d62a-803b-4217-8067-c2077da6cc35 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63de41f4348243998029f7d07aa5dc40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:09:43.839149 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0296d62a-803b-4217-8067-c2077da6cc35 service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:43.839418 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0296d62a-803b-4217-8067-c2077da6cc35 service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:43.839681 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0296d62a-803b-4217-8067-c2077da6cc35 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:43.841407 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0296d62a-803b-4217-8067-c2077da6cc35 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:43.942463 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0296d62a-803b-4217-8067-c2077da6cc35 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:09:43.945396 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0296d62a-803b-4217-8067-c2077da6cc35 service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:09:43.971545 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 718/1419] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:09:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:09:44.867139 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4146664-f464-4f5f-9575-d4f8078d29cb None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:44.867719 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4146664-f464-4f5f-9575-d4f8078d29cb None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:44.867719 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4146664-f464-4f5f-9575-d4f8078d29cb None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:44.869088 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 702/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 16:09:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:09:44.875179 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c1b3b-87db-49cc-9223-475ba67b8ac6 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:44.875825 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c1b3b-87db-49cc-9223-475ba67b8ac6 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:44.876060 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c1b3b-87db-49cc-9223-475ba67b8ac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:44.948118 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:09:44.978444 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-2f6c1b3b-87db-49cc-9223-475ba67b8ac6 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:09:44.979755 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-2f6c1b3b-87db-49cc-9223-475ba67b8ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:45.081681 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 719/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 16:09:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:09:45.235958 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4146664-f464-4f5f-9575-d4f8078d29cb None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:09:45.238464 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f4146664-f464-4f5f-9575-d4f8078d29cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:45.386611 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4146664-f464-4f5f-9575-d4f8078d29cb None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:09:45.399658 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:09:45.506836 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c113ff8-5232-4da8-b797-363ce9c190d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63de41f4348243998029f7d07aa5dc40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:09:45.507383 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c113ff8-5232-4da8-b797-363ce9c190d2 service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:45.507658 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c113ff8-5232-4da8-b797-363ce9c190d2 service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:45.507932 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c113ff8-5232-4da8-b797-363ce9c190d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:45.509579 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1c113ff8-5232-4da8-b797-363ce9c190d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:45.608256 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c113ff8-5232-4da8-b797-363ce9c190d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:09:45.609889 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c113ff8-5232-4da8-b797-363ce9c190d2 service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:09:45.636132 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 703/1422] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:09:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:09:46.501259 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c551da7-a97e-4e2b-a087-29f4435076c6 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:46.502145 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c551da7-a97e-4e2b-a087-29f4435076c6 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:46.502145 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c551da7-a97e-4e2b-a087-29f4435076c6 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:46.504023 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 720/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 16:09:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:09:46.509234 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9350be2-dbec-42c8-b88d-abc99dea4a0a None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:46.509657 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9350be2-dbec-42c8-b88d-abc99dea4a0a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:46.509747 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9350be2-dbec-42c8-b88d-abc99dea4a0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:46.530073 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:09:46.588947 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-a9350be2-dbec-42c8-b88d-abc99dea4a0a None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:09:46.590146 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a9350be2-dbec-42c8-b88d-abc99dea4a0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:46.689348 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 704/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 16:09:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:09:46.848974 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c551da7-a97e-4e2b-a087-29f4435076c6 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:09:46.851367 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5c551da7-a97e-4e2b-a087-29f4435076c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:46.974757 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:09:47.012068 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c551da7-a97e-4e2b-a087-29f4435076c6 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:09:47.144236 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7ffd1e-26b4-4c2d-88df-e4cd4ddc5b91 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63de41f4348243998029f7d07aa5dc40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:09:47.144771 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ffd1e-26b4-4c2d-88df-e4cd4ddc5b91 service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:47.145075 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ffd1e-26b4-4c2d-88df-e4cd4ddc5b91 service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:47.145342 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ffd1e-26b4-4c2d-88df-e4cd4ddc5b91 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:47.146946 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7d7ffd1e-26b4-4c2d-88df-e4cd4ddc5b91 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:47.238072 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:09:47.253714 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7ffd1e-26b4-4c2d-88df-e4cd4ddc5b91 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:09:47.255437 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7ffd1e-26b4-4c2d-88df-e4cd4ddc5b91 service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:09:47.274095 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 721/1425] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:09:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:09:48.125907 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd73f2-d61c-4615-96fb-147fe0a4768e None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:48.126512 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd73f2-d61c-4615-96fb-147fe0a4768e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:48.126512 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd73f2-d61c-4615-96fb-147fe0a4768e None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:48.127808 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 705/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 16:09:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:09:48.134420 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efa24ef-fb74-4d76-913e-67d38581ff92 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:48.135177 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efa24ef-fb74-4d76-913e-67d38581ff92 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:48.135545 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efa24ef-fb74-4d76-913e-67d38581ff92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:48.232562 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-2efa24ef-fb74-4d76-913e-67d38581ff92 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:09:48.234408 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-2efa24ef-fb74-4d76-913e-67d38581ff92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:48.343736 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 722/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 16:09:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:09:48.521212 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdd73f2-d61c-4615-96fb-147fe0a4768e None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:09:48.523547 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-dfdd73f2-d61c-4615-96fb-147fe0a4768e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:48.618237 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:09:48.690365 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdd73f2-d61c-4615-96fb-147fe0a4768e None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:09:48.819410 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8200c413-19ba-414d-a01a-f41351c13e63 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63de41f4348243998029f7d07aa5dc40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:09:48.820019 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8200c413-19ba-414d-a01a-f41351c13e63 service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:48.820290 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8200c413-19ba-414d-a01a-f41351c13e63 service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:48.820557 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8200c413-19ba-414d-a01a-f41351c13e63 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:48.822509 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-8200c413-19ba-414d-a01a-f41351c13e63 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:48.871037 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:09:48.913376 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8200c413-19ba-414d-a01a-f41351c13e63 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:09:48.915019 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8200c413-19ba-414d-a01a-f41351c13e63 service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:09:48.935120 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 706/1428] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:09:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:09:49.742679 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6196687c-105c-43f4-a165-01deaa3fd764 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:49.743532 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6196687c-105c-43f4-a165-01deaa3fd764 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:49.743532 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6196687c-105c-43f4-a165-01deaa3fd764 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:49.745381 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 723/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 16:09:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:09:49.750567 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569a16b2-ea28-4933-9073-b300f261b938 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:49.750991 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569a16b2-ea28-4933-9073-b300f261b938 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:49.751143 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569a16b2-ea28-4933-9073-b300f261b938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:49.827809 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-569a16b2-ea28-4933-9073-b300f261b938 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:09:49.829046 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-569a16b2-ea28-4933-9073-b300f261b938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:49.934925 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 707/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 16:09:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:09:50.090337 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6196687c-105c-43f4-a165-01deaa3fd764 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:09:50.092700 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-6196687c-105c-43f4-a165-01deaa3fd764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:50.148589 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:09:50.255789 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6196687c-105c-43f4-a165-01deaa3fd764 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:09:50.385277 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a7d9de-b4f4-48da-997e-87a8177ffabf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63de41f4348243998029f7d07aa5dc40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:09:50.385831 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a7d9de-b4f4-48da-997e-87a8177ffabf service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:50.386106 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a7d9de-b4f4-48da-997e-87a8177ffabf service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:50.386370 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a7d9de-b4f4-48da-997e-87a8177ffabf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:50.387962 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-30a7d9de-b4f4-48da-997e-87a8177ffabf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:50.394916 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:09:50.491193 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a7d9de-b4f4-48da-997e-87a8177ffabf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:09:50.492839 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a7d9de-b4f4-48da-997e-87a8177ffabf service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:09:50.512409 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 724/1431] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:09:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:09:51.318064 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77d5e88-b1ee-479a-9f5e-29f3bc3dd8ca None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:51.318565 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77d5e88-b1ee-479a-9f5e-29f3bc3dd8ca None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:51.318668 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77d5e88-b1ee-479a-9f5e-29f3bc3dd8ca None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:51.320088 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 708/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 16:09:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:09:51.328409 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb437994-3783-4a14-9158-928acb2e19f9 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:51.328892 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb437994-3783-4a14-9158-928acb2e19f9 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:51.329001 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb437994-3783-4a14-9158-928acb2e19f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:51.408867 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-bb437994-3783-4a14-9158-928acb2e19f9 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:09:51.410085 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-bb437994-3783-4a14-9158-928acb2e19f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:51.483904 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:09:51.513903 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 725/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 16:09:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:09:51.662232 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77d5e88-b1ee-479a-9f5e-29f3bc3dd8ca None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:09:51.664559 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-c77d5e88-b1ee-479a-9f5e-29f3bc3dd8ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:51.685804 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:09:51.822319 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77d5e88-b1ee-479a-9f5e-29f3bc3dd8ca None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:09:51.924160 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:09:51.950867 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9d6ec8-0f2b-4b45-830b-7aa488778cd7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63de41f4348243998029f7d07aa5dc40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:09:51.951505 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9d6ec8-0f2b-4b45-830b-7aa488778cd7 service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:51.951784 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9d6ec8-0f2b-4b45-830b-7aa488778cd7 service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:51.952057 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9d6ec8-0f2b-4b45-830b-7aa488778cd7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:51.953696 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-df9d6ec8-0f2b-4b45-830b-7aa488778cd7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:52.055809 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9d6ec8-0f2b-4b45-830b-7aa488778cd7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:09:52.057587 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9d6ec8-0f2b-4b45-830b-7aa488778cd7 service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:09:52.075749 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 709/1434] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:09:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:09:52.914336 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed4110a-e252-434a-9f51-50cf3eedf6c9 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:52.915272 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed4110a-e252-434a-9f51-50cf3eedf6c9 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:52.915272 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed4110a-e252-434a-9f51-50cf3eedf6c9 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:52.917563 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 726/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 16:09:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:09:52.925932 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82da14a-4caf-49c7-9ac5-a67484969072 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:52.926368 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82da14a-4caf-49c7-9ac5-a67484969072 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:52.926504 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82da14a-4caf-49c7-9ac5-a67484969072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:53.008895 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-a82da14a-4caf-49c7-9ac5-a67484969072 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:09:53.010111 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a82da14a-4caf-49c7-9ac5-a67484969072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:53.056494 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:09:53.123221 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 710/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 16:09:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:09:53.270039 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed4110a-e252-434a-9f51-50cf3eedf6c9 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:09:53.271585 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-3ed4110a-e252-434a-9f51-50cf3eedf6c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:53.432329 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed4110a-e252-434a-9f51-50cf3eedf6c9 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:09:53.487516 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:09:53.546928 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca631924-ed91-494b-96cb-064f2dbdb0b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63de41f4348243998029f7d07aa5dc40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:09:53.547458 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca631924-ed91-494b-96cb-064f2dbdb0b1 service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:53.547725 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca631924-ed91-494b-96cb-064f2dbdb0b1 service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:53.547990 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca631924-ed91-494b-96cb-064f2dbdb0b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:53.549606 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ca631924-ed91-494b-96cb-064f2dbdb0b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:53.642566 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca631924-ed91-494b-96cb-064f2dbdb0b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:09:53.644153 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca631924-ed91-494b-96cb-064f2dbdb0b1 service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:09:53.662084 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 727/1437] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:09:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:09:54.481147 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79a099d-6eca-4bce-b450-0e6efabd2720 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:54.481742 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79a099d-6eca-4bce-b450-0e6efabd2720 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:54.481742 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79a099d-6eca-4bce-b450-0e6efabd2720 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:54.483803 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 711/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 16:09:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:09:54.492124 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9547a485-b285-4d87-aeb8-a710684cf90c None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:54.492548 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9547a485-b285-4d87-aeb8-a710684cf90c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:54.492746 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9547a485-b285-4d87-aeb8-a710684cf90c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:54.572682 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-9547a485-b285-4d87-aeb8-a710684cf90c None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:09:54.573865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-9547a485-b285-4d87-aeb8-a710684cf90c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:54.575965 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:09:54.676247 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 728/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 16:09:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:09:54.840096 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79a099d-6eca-4bce-b450-0e6efabd2720 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:09:54.842591 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f79a099d-6eca-4bce-b450-0e6efabd2720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:55.001549 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79a099d-6eca-4bce-b450-0e6efabd2720 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:09:55.030758 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:09:55.132146 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1103b5e1-68f5-4ca0-ae58-c4e23a21b0e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63de41f4348243998029f7d07aa5dc40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:09:55.132683 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1103b5e1-68f5-4ca0-ae58-c4e23a21b0e7 service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:55.132972 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1103b5e1-68f5-4ca0-ae58-c4e23a21b0e7 service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:55.133261 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1103b5e1-68f5-4ca0-ae58-c4e23a21b0e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:55.134858 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1103b5e1-68f5-4ca0-ae58-c4e23a21b0e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:55.226573 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1103b5e1-68f5-4ca0-ae58-c4e23a21b0e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:09:55.228200 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1103b5e1-68f5-4ca0-ae58-c4e23a21b0e7 service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:09:55.246888 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 712/1440] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:09:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:09:57.050339 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd6247-ae84-4e59-82dc-440cd9fec91e None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:57.050969 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd6247-ae84-4e59-82dc-440cd9fec91e None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:57.050969 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd6247-ae84-4e59-82dc-440cd9fec91e None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:57.052249 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 729/1441] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 16:09:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:09:57.061749 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d0d08-48db-4299-a8cb-b3eadeec8e65 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:57.062204 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d0d08-48db-4299-a8cb-b3eadeec8e65 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:57.062316 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d0d08-48db-4299-a8cb-b3eadeec8e65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:57.109061 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:09:57.136726 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-b77d0d08-48db-4299-a8cb-b3eadeec8e65 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:09:57.138003 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b77d0d08-48db-4299-a8cb-b3eadeec8e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:57.246839 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 713/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 16:09:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:09:57.526829 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fd6247-ae84-4e59-82dc-440cd9fec91e None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:09:57.528392 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e0fd6247-ae84-4e59-82dc-440cd9fec91e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:57.638004 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fd6247-ae84-4e59-82dc-440cd9fec91e None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:09:57.670195 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:09:57.717287 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9cbf72-0987-42ea-8607-94d3d5362ae4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:09:57.717904 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9cbf72-0987-42ea-8607-94d3d5362ae4 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:57.718182 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9cbf72-0987-42ea-8607-94d3d5362ae4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:57.718450 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9cbf72-0987-42ea-8607-94d3d5362ae4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:57.720060 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-fe9cbf72-0987-42ea-8607-94d3d5362ae4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:57.838561 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9cbf72-0987-42ea-8607-94d3d5362ae4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:09:57.840063 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9cbf72-0987-42ea-8607-94d3d5362ae4 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:09:57.859006 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 730/1443] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:09:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:09:59.953807 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45ebb6-12c5-4e5a-a174-a011db869b49 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:59.954385 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45ebb6-12c5-4e5a-a174-a011db869b49 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:59.954385 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45ebb6-12c5-4e5a-a174-a011db869b49 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:59.955610 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 714/1444] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 16:09:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:09:59.961364 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397dade8-3b4c-4928-b285-a780ec8f2e9c None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:59.961794 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397dade8-3b4c-4928-b285-a780ec8f2e9c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:59.961883 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397dade8-3b4c-4928-b285-a780ec8f2e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:10:00.031287 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-397dade8-3b4c-4928-b285-a780ec8f2e9c None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:10:00.032489 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-397dade8-3b4c-4928-b285-a780ec8f2e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:10:00.045763 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:10:00.139346 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 731/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 16:09:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:10:00.422092 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a45ebb6-12c5-4e5a-a174-a011db869b49 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:10:00.424549 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3a45ebb6-12c5-4e5a-a174-a011db869b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:10:00.545002 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a45ebb6-12c5-4e5a-a174-a011db869b49 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:10:00.557189 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:10:00.623395 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7ee967-bfda-4efe-bbf9-3cac1a2e82dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:10:00.623992 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7ee967-bfda-4efe-bbf9-3cac1a2e82dd service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:10:00.624266 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7ee967-bfda-4efe-bbf9-3cac1a2e82dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:10:00.624529 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7ee967-bfda-4efe-bbf9-3cac1a2e82dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:10:00.626139 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6b7ee967-bfda-4efe-bbf9-3cac1a2e82dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:10:00.740992 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7ee967-bfda-4efe-bbf9-3cac1a2e82dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:10:00.742439 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7ee967-bfda-4efe-bbf9-3cac1a2e82dd service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:10:00.761850 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 715/1446] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:10:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:10:03.009015 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4979a29-0b70-4edc-ae0e-d1b55247b410 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:10:03.009623 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4979a29-0b70-4edc-ae0e-d1b55247b410 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:10:03.009623 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4979a29-0b70-4edc-ae0e-d1b55247b410 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:10:03.010925 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 732/1447] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 16:10:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:10:03.017406 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b76acb5-eb37-4e94-8b9c-5ed539360510 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:10:03.017849 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b76acb5-eb37-4e94-8b9c-5ed539360510 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:10:03.017972 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b76acb5-eb37-4e94-8b9c-5ed539360510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:10:03.079412 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:10:03.101458 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-2b76acb5-eb37-4e94-8b9c-5ed539360510 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:10:03.102652 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-2b76acb5-eb37-4e94-8b9c-5ed539360510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:10:03.204501 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 716/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 16:10:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:10:03.481711 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4979a29-0b70-4edc-ae0e-d1b55247b410 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:10:03.483362 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c4979a29-0b70-4edc-ae0e-d1b55247b410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:10:03.592643 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4979a29-0b70-4edc-ae0e-d1b55247b410 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:10:03.633265 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:10:03.668989 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4896caf-07a8-49f0-b5d2-19cb0c058081 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:10:03.669599 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4896caf-07a8-49f0-b5d2-19cb0c058081 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:10:03.669869 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4896caf-07a8-49f0-b5d2-19cb0c058081 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:10:03.670136 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4896caf-07a8-49f0-b5d2-19cb0c058081 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:10:03.671724 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c4896caf-07a8-49f0-b5d2-19cb0c058081 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:10:03.789202 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4896caf-07a8-49f0-b5d2-19cb0c058081 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:10:03.790682 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4896caf-07a8-49f0-b5d2-19cb0c058081 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:10:03.810515 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 733/1449] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:10:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:10:05.855506 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab2137-b588-4ec9-b328-a508d988df32 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:10:05.856388 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab2137-b588-4ec9-b328-a508d988df32 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:10:05.856388 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab2137-b588-4ec9-b328-a508d988df32 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:10:05.858688 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 717/1450] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 16:10:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:10:05.864135 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a6bfda-6c25-4cd3-8bc4-6d865040c3e4 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:10:05.864611 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a6bfda-6c25-4cd3-8bc4-6d865040c3e4 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:10:05.864716 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a6bfda-6c25-4cd3-8bc4-6d865040c3e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:10:05.935261 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-b5a6bfda-6c25-4cd3-8bc4-6d865040c3e4 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:10:05.936485 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b5a6bfda-6c25-4cd3-8bc4-6d865040c3e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:10:05.959142 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:10:06.038511 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 734/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 16:10:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:10:06.327724 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ab2137-b588-4ec9-b328-a508d988df32 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:10:06.330367 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-37ab2137-b588-4ec9-b328-a508d988df32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:10:06.447422 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ab2137-b588-4ec9-b328-a508d988df32 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:10:06.467316 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:10:06.524003 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f10a43-f463-4f8a-8097-ea3b3a6f486f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:10:06.524616 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f10a43-f463-4f8a-8097-ea3b3a6f486f service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:10:06.524927 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f10a43-f463-4f8a-8097-ea3b3a6f486f service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:10:06.525192 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f10a43-f463-4f8a-8097-ea3b3a6f486f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:10:06.526792 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b9f10a43-f463-4f8a-8097-ea3b3a6f486f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:10:06.648699 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f10a43-f463-4f8a-8097-ea3b3a6f486f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:10:06.650103 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f10a43-f463-4f8a-8097-ea3b3a6f486f service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:10:06.669174 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 718/1452] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:10:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:10:08.639436 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c8c0a8-8378-4d35-becd-37e17ddf32a1 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:10:08.640289 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c8c0a8-8378-4d35-becd-37e17ddf32a1 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:10:08.640289 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c8c0a8-8378-4d35-becd-37e17ddf32a1 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:10:08.642475 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 735/1453] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 16:10:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:10:08.649081 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f4010-d7d7-4fb5-a59f-9b8cb151696b None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:10:08.649811 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f4010-d7d7-4fb5-a59f-9b8cb151696b None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:10:08.649910 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f4010-d7d7-4fb5-a59f-9b8cb151696b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:10:08.717534 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:10:08.720407 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-e11f4010-d7d7-4fb5-a59f-9b8cb151696b None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:10:08.722491 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e11f4010-d7d7-4fb5-a59f-9b8cb151696b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:10:08.823083 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 719/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 16:10:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:10:09.111189 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c8c0a8-8378-4d35-becd-37e17ddf32a1 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:10:09.113656 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-18c8c0a8-8378-4d35-becd-37e17ddf32a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:10:09.229401 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c8c0a8-8378-4d35-becd-37e17ddf32a1 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:10:09.281681 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:10:09.308337 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c8c7a8-a18d-4d37-a46a-5138904150f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:10:09.308978 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c8c7a8-a18d-4d37-a46a-5138904150f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:10:09.309252 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c8c7a8-a18d-4d37-a46a-5138904150f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:10:09.309522 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c8c7a8-a18d-4d37-a46a-5138904150f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:10:09.311199 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d6c8c7a8-a18d-4d37-a46a-5138904150f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:10:09.420547 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c8c7a8-a18d-4d37-a46a-5138904150f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:10:09.421938 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c8c7a8-a18d-4d37-a46a-5138904150f5 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:10:09.440418 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 736/1455] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:10:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:10:11.422760 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f112eb2c-6b04-4ddc-98d2-281364746e93 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:10:11.423351 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f112eb2c-6b04-4ddc-98d2-281364746e93 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:10:11.423351 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f112eb2c-6b04-4ddc-98d2-281364746e93 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:10:11.424665 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 720/1456] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 16:10:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:10:11.431253 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e71a0-5709-48ac-ad3c-8c345d69514a None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:10:11.431975 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e71a0-5709-48ac-ad3c-8c345d69514a None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:10:11.432122 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e71a0-5709-48ac-ad3c-8c345d69514a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:10:11.501721 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-ea6e71a0-5709-48ac-ad3c-8c345d69514a None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:10:11.502945 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ea6e71a0-5709-48ac-ad3c-8c345d69514a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:10:11.533392 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:10:11.600658 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 737/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 16:10:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:10:11.885913 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f112eb2c-6b04-4ddc-98d2-281364746e93 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:10:11.887651 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f112eb2c-6b04-4ddc-98d2-281364746e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:10:12.005131 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f112eb2c-6b04-4ddc-98d2-281364746e93 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:10:12.033870 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:10:12.083568 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6dc5f1-297e-4a89-94d3-819c8a0d0331 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:10:12.084193 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6dc5f1-297e-4a89-94d3-819c8a0d0331 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:10:12.084446 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6dc5f1-297e-4a89-94d3-819c8a0d0331 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:10:12.084796 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6dc5f1-297e-4a89-94d3-819c8a0d0331 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:10:12.086400 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-fa6dc5f1-297e-4a89-94d3-819c8a0d0331 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:10:12.209129 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6dc5f1-297e-4a89-94d3-819c8a0d0331 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:10:12.210486 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6dc5f1-297e-4a89-94d3-819c8a0d0331 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:10:12.228905 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 721/1458] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:10:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:10:14.182810 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528a0668-1bcb-4885-b069-018ec4220c7b None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:10:14.183648 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528a0668-1bcb-4885-b069-018ec4220c7b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:10:14.183648 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528a0668-1bcb-4885-b069-018ec4220c7b None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:10:14.185769 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 738/1459] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 16:10:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:10:14.192207 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4540a9-84ba-4e3e-acd6-e84b0423807d None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:10:14.192922 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4540a9-84ba-4e3e-acd6-e84b0423807d None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:10:14.193135 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4540a9-84ba-4e3e-acd6-e84b0423807d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:10:14.265072 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-1d4540a9-84ba-4e3e-acd6-e84b0423807d None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:10:14.266293 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1d4540a9-84ba-4e3e-acd6-e84b0423807d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:10:14.275016 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:10:14.367365 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 722/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 16:10:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:10:14.661567 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528a0668-1bcb-4885-b069-018ec4220c7b None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:10:14.664010 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-528a0668-1bcb-4885-b069-018ec4220c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:10:14.778606 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528a0668-1bcb-4885-b069-018ec4220c7b None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:10:14.835404 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:10:14.861172 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe8621a-45d1-410d-a3b8-9e6a7c15247e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:10:14.861772 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe8621a-45d1-410d-a3b8-9e6a7c15247e service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:10:14.862045 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe8621a-45d1-410d-a3b8-9e6a7c15247e service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:10:14.862310 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe8621a-45d1-410d-a3b8-9e6a7c15247e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:10:14.863898 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-bbe8621a-45d1-410d-a3b8-9e6a7c15247e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:10:14.981094 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe8621a-45d1-410d-a3b8-9e6a7c15247e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:10:14.982465 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe8621a-45d1-410d-a3b8-9e6a7c15247e service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:10:15.001154 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 739/1461] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:10:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:10:16.958022 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a382a96c-6561-4de9-bc84-bfca16b266bc None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:10:16.958619 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a382a96c-6561-4de9-bc84-bfca16b266bc None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:10:16.958732 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a382a96c-6561-4de9-bc84-bfca16b266bc None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:10:16.961071 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 723/1462] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 16:10:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:10:16.966702 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee4231b-c8a7-47e3-bba1-7d668e0d6c01 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:10:16.967151 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee4231b-c8a7-47e3-bba1-7d668e0d6c01 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:10:16.967257 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee4231b-c8a7-47e3-bba1-7d668e0d6c01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:10:17.043147 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-aee4231b-c8a7-47e3-bba1-7d668e0d6c01 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:10:17.044365 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-aee4231b-c8a7-47e3-bba1-7d668e0d6c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:10:17.087651 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:10:17.151929 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 740/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 16:10:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:10:17.439088 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a382a96c-6561-4de9-bc84-bfca16b266bc None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:10:17.441504 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a382a96c-6561-4de9-bc84-bfca16b266bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:10:17.553447 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a382a96c-6561-4de9-bc84-bfca16b266bc None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:10:17.591595 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:10:17.631993 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb8b401-d092-437d-b17e-3cce53a25418 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:10:17.632602 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb8b401-d092-437d-b17e-3cce53a25418 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:10:17.632965 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb8b401-d092-437d-b17e-3cce53a25418 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:10:17.633251 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb8b401-d092-437d-b17e-3cce53a25418 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:10:17.634832 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-fcb8b401-d092-437d-b17e-3cce53a25418 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:10:17.754553 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb8b401-d092-437d-b17e-3cce53a25418 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:10:17.756538 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb8b401-d092-437d-b17e-3cce53a25418 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:10:17.781973 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 724/1464] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:10:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:10:19.712645 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c83d1-f56e-4883-b3d1-766dc3280695 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:10:19.713350 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c83d1-f56e-4883-b3d1-766dc3280695 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:10:19.713350 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c83d1-f56e-4883-b3d1-766dc3280695 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:10:19.714607 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 741/1465] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 16:10:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:10:19.720048 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df127a13-0cf4-4000-8555-4ff4a50af76b None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:10:19.720510 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df127a13-0cf4-4000-8555-4ff4a50af76b None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:10:19.720622 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df127a13-0cf4-4000-8555-4ff4a50af76b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:10:19.792108 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-df127a13-0cf4-4000-8555-4ff4a50af76b None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:10:19.793363 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-df127a13-0cf4-4000-8555-4ff4a50af76b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:10:19.812791 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:10:19.898610 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 725/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 16:10:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:10:20.187186 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23c83d1-f56e-4883-b3d1-766dc3280695 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:10:20.189807 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f23c83d1-f56e-4883-b3d1-766dc3280695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:10:20.307323 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23c83d1-f56e-4883-b3d1-766dc3280695 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:10:20.375528 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:10:20.383705 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa734666-caf4-4210-9959-9636e5891ca8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:10:20.384323 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa734666-caf4-4210-9959-9636e5891ca8 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:10:20.384600 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa734666-caf4-4210-9959-9636e5891ca8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:10:20.384974 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa734666-caf4-4210-9959-9636e5891ca8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:10:20.386721 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-aa734666-caf4-4210-9959-9636e5891ca8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:10:20.525965 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa734666-caf4-4210-9959-9636e5891ca8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:10:20.527571 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa734666-caf4-4210-9959-9636e5891ca8 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:10:20.547036 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 742/1467] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:10:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:10:22.463728 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c9e393-324d-4187-b94c-4839c22f38c1 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:10:22.464629 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c9e393-324d-4187-b94c-4839c22f38c1 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:10:22.464629 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c9e393-324d-4187-b94c-4839c22f38c1 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:10:22.466965 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 726/1468] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 16:10:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:10:22.473607 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2752ca6f-10a2-4743-b7a0-471bcf1e509a None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:10:22.474084 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2752ca6f-10a2-4743-b7a0-471bcf1e509a None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:10:22.474189 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2752ca6f-10a2-4743-b7a0-471bcf1e509a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:10:22.554072 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-2752ca6f-10a2-4743-b7a0-471bcf1e509a None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:10:22.555235 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-2752ca6f-10a2-4743-b7a0-471bcf1e509a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:10:22.610789 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:10:22.662350 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 743/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 16:10:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:10:22.943129 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c9e393-324d-4187-b94c-4839c22f38c1 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:10:22.945952 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-87c9e393-324d-4187-b94c-4839c22f38c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:10:23.064220 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c9e393-324d-4187-b94c-4839c22f38c1 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:10:23.125094 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:10:23.157151 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5715e92a-0e60-4f77-8535-7ba4a478d0fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:10:23.157759 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5715e92a-0e60-4f77-8535-7ba4a478d0fc service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:10:23.158037 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5715e92a-0e60-4f77-8535-7ba4a478d0fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:10:23.158298 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5715e92a-0e60-4f77-8535-7ba4a478d0fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:10:23.159898 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5715e92a-0e60-4f77-8535-7ba4a478d0fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:10:23.284380 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5715e92a-0e60-4f77-8535-7ba4a478d0fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:10:23.285857 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5715e92a-0e60-4f77-8535-7ba4a478d0fc service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:10:23.305906 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 727/1470] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:10:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:10:32.660565 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=22 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:10:33.303653 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=66 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:10:42.654508 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af67e6a-b910-43c1-b7f5-8b53f54a14c8 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:10:42.655146 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af67e6a-b910-43c1-b7f5-8b53f54a14c8 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:10:42.655146 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af67e6a-b910-43c1-b7f5-8b53f54a14c8 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:10:42.656336 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 744/1471] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:10:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:10:42.662839 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd982c-e4cc-488f-a993-f61c72c567e1 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:10:42.663591 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd982c-e4cc-488f-a993-f61c72c567e1 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:10:42.663744 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd982c-e4cc-488f-a993-f61c72c567e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:10:42.734125 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-a6dd982c-e4cc-488f-a993-f61c72c567e1 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:10:42.735355 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a6dd982c-e4cc-488f-a993-f61c72c567e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:10:42.833543 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 728/1472] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:10:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:10:43.127788 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f8d4f6-f4ad-4780-a138-645e43a90ed2 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:10:43.128082 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f8d4f6-f4ad-4780-a138-645e43a90ed2 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:10:43.128347 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f8d4f6-f4ad-4780-a138-645e43a90ed2 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:10:43.129730 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 745/1473] 10.222.0.43 () {54 vars in 961 bytes} [Tue Sep 6 16:10:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:10:43.134844 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221730df-4479-4ce7-b52f-4c8ddd68302a None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:10:43.135125 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221730df-4479-4ce7-b52f-4c8ddd68302a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:10:43.135393 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221730df-4479-4ce7-b52f-4c8ddd68302a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:10:43.213995 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-221730df-4479-4ce7-b52f-4c8ddd68302a None None] MFA Rules not processed for user `2c163b39e61648c385f065ce8091c3ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:10:43.215220 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-221730df-4479-4ce7-b52f-4c8ddd68302a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:10:43.237125 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:10:43.298271 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 729/1474] 10.222.0.43 () {58 vars in 1058 bytes} [Tue Sep 6 16:10:43 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:10:43.310280 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f8d4f6-f4ad-4780-a138-645e43a90ed2 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:10:43.312888 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-58f8d4f6-f4ad-4780-a138-645e43a90ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:10:43.418580 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f8d4f6-f4ad-4780-a138-645e43a90ed2 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:10:43.493515 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3339a5-b558-4403-8993-8677b7311c94 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c163b39e61648c385f065ce8091c3ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:10:43.494122 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3339a5-b558-4403-8993-8677b7311c94 service glance] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:10:43.494426 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3339a5-b558-4403-8993-8677b7311c94 service glance] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:10:43.494696 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3339a5-b558-4403-8993-8677b7311c94 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:10:43.496286 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ab3339a5-b558-4403-8993-8677b7311c94 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:10:43.551369 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:10:43.609111 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3339a5-b558-4403-8993-8677b7311c94 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:10:43.610616 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3339a5-b558-4403-8993-8677b7311c94 service glance] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:10:43.629481 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 746/1475] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:10:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:10:45.665749 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc70f49-f621-456a-b8a8-e2a8d78dce94 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:10:45.666721 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc70f49-f621-456a-b8a8-e2a8d78dce94 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:10:45.666721 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc70f49-f621-456a-b8a8-e2a8d78dce94 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:10:45.668611 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 730/1476] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:10:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:10:45.677430 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e1a084-12c9-48df-9942-4fb67e5fcaee None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:10:45.677870 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e1a084-12c9-48df-9942-4fb67e5fcaee None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:10:45.677995 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e1a084-12c9-48df-9942-4fb67e5fcaee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:10:45.756286 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-f1e1a084-12c9-48df-9942-4fb67e5fcaee None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:10:45.757516 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f1e1a084-12c9-48df-9942-4fb67e5fcaee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:10:45.860770 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 747/1477] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:10:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:10:46.129000 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc70f49-f621-456a-b8a8-e2a8d78dce94 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:10:46.130529 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6bc70f49-f621-456a-b8a8-e2a8d78dce94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:10:46.265332 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc70f49-f621-456a-b8a8-e2a8d78dce94 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:10:46.317137 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:10:46.366137 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5706431e-8b06-4271-a712-fa3d6d851fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:10:46.366751 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5706431e-8b06-4271-a712-fa3d6d851fed admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:10:46.367030 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5706431e-8b06-4271-a712-fa3d6d851fed admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:10:46.367295 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5706431e-8b06-4271-a712-fa3d6d851fed admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:10:46.367934 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5706431e-8b06-4271-a712-fa3d6d851fed admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:10:46.369837 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: WARNING py.warnings [None req-5706431e-8b06-4271-a712-fa3d6d851fed admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 16:10:46.369837 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: warnings.warn(msg) Sep 06 16:10:46.369837 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: Sep 06 16:10:46.370993 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5706431e-8b06-4271-a712-fa3d6d851fed admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:10:46.389616 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 731/1478] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 6 16:10:46 2022] GET /identity/v3/projects => generated 2553 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:10:48.289517 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27e854b-15cf-41f7-a5e1-dd3ee7863a58 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:10:48.290258 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27e854b-15cf-41f7-a5e1-dd3ee7863a58 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:10:48.290258 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27e854b-15cf-41f7-a5e1-dd3ee7863a58 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:10:48.292544 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 748/1479] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:10:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:10:48.298778 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d2b73e-df4b-42b7-a79d-91cf6fa96a3a None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:10:48.299245 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d2b73e-df4b-42b7-a79d-91cf6fa96a3a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:10:48.299358 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d2b73e-df4b-42b7-a79d-91cf6fa96a3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:10:48.374325 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-55d2b73e-df4b-42b7-a79d-91cf6fa96a3a None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:10:48.375568 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-55d2b73e-df4b-42b7-a79d-91cf6fa96a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:10:48.473093 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 732/1480] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:10:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:10:48.775403 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27e854b-15cf-41f7-a5e1-dd3ee7863a58 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:10:48.777063 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d27e854b-15cf-41f7-a5e1-dd3ee7863a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:10:48.787153 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:10:48.930203 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27e854b-15cf-41f7-a5e1-dd3ee7863a58 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:10:49.021558 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:10:49.055863 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c685e0d-03bf-4b83-8eb3-20ad66e063de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63de41f4348243998029f7d07aa5dc40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:10:49.056466 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c685e0d-03bf-4b83-8eb3-20ad66e063de service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:10:49.056735 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c685e0d-03bf-4b83-8eb3-20ad66e063de service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:10:49.057030 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c685e0d-03bf-4b83-8eb3-20ad66e063de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:10:49.058705 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-4c685e0d-03bf-4b83-8eb3-20ad66e063de service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:10:49.153318 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c685e0d-03bf-4b83-8eb3-20ad66e063de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:10:49.154905 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c685e0d-03bf-4b83-8eb3-20ad66e063de service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:10:49.174354 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 749/1481] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:10:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:10:51.305991 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444a4c92-ba13-4425-bf21-b195928e458d None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:10:51.306549 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444a4c92-ba13-4425-bf21-b195928e458d None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:10:51.306549 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444a4c92-ba13-4425-bf21-b195928e458d None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:10:51.308805 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 733/1482] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:10:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:10:51.313732 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090f8554-04cd-4b65-8bc8-862c17e5239a None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:10:51.314249 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090f8554-04cd-4b65-8bc8-862c17e5239a None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:10:51.314353 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090f8554-04cd-4b65-8bc8-862c17e5239a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:10:51.383768 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-090f8554-04cd-4b65-8bc8-862c17e5239a None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:10:51.385018 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-090f8554-04cd-4b65-8bc8-862c17e5239a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:10:51.430518 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:10:51.489177 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 750/1483] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:10:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:10:51.782702 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444a4c92-ba13-4425-bf21-b195928e458d None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:10:51.784324 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-444a4c92-ba13-4425-bf21-b195928e458d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:10:51.813204 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:10:51.895065 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444a4c92-ba13-4425-bf21-b195928e458d None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:10:51.968961 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39fd382-eb17-4ef5-b06c-385a4e592f0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:10:51.969559 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39fd382-eb17-4ef5-b06c-385a4e592f0d service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:10:51.969832 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39fd382-eb17-4ef5-b06c-385a4e592f0d service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:10:51.970097 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39fd382-eb17-4ef5-b06c-385a4e592f0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:10:51.971694 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d39fd382-eb17-4ef5-b06c-385a4e592f0d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:10:52.051035 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:10:52.090348 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39fd382-eb17-4ef5-b06c-385a4e592f0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:10:52.091892 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39fd382-eb17-4ef5-b06c-385a4e592f0d service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:10:52.113485 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 734/1484] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:10:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:10:54.060510 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e803c8d-7609-4a6d-b8a3-fad89fbd2542 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:10:54.061079 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e803c8d-7609-4a6d-b8a3-fad89fbd2542 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:10:54.061079 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e803c8d-7609-4a6d-b8a3-fad89fbd2542 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:10:54.062435 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 751/1485] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:10:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:10:54.068162 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49068c3-5a30-4e62-921a-ab0b0eee8590 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:10:54.068646 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49068c3-5a30-4e62-921a-ab0b0eee8590 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:10:54.068789 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49068c3-5a30-4e62-921a-ab0b0eee8590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:10:54.147591 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-f49068c3-5a30-4e62-921a-ab0b0eee8590 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:10:54.149542 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f49068c3-5a30-4e62-921a-ab0b0eee8590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:10:54.257765 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 735/1486] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:10:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:10:54.560251 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e803c8d-7609-4a6d-b8a3-fad89fbd2542 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:10:54.562729 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-3e803c8d-7609-4a6d-b8a3-fad89fbd2542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:10:54.682042 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e803c8d-7609-4a6d-b8a3-fad89fbd2542 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:10:54.762475 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:10:54.765601 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc2b047-73a9-40e6-98cb-0a8e3a021955 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:10:54.766499 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc2b047-73a9-40e6-98cb-0a8e3a021955 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:10:54.766774 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc2b047-73a9-40e6-98cb-0a8e3a021955 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:10:54.767886 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc2b047-73a9-40e6-98cb-0a8e3a021955 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:10:54.769795 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-2dc2b047-73a9-40e6-98cb-0a8e3a021955 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:10:54.896883 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc2b047-73a9-40e6-98cb-0a8e3a021955 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:10:54.898268 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc2b047-73a9-40e6-98cb-0a8e3a021955 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:10:54.916907 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 752/1487] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:10:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:10:56.960190 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062f4310-126d-4f3a-817d-1163da679927 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:10:56.961277 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062f4310-126d-4f3a-817d-1163da679927 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:10:56.961277 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062f4310-126d-4f3a-817d-1163da679927 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:10:56.963055 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 736/1488] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:10:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:10:56.969321 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab24b3-e158-47e6-9eb6-527377ed3302 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:10:56.970070 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab24b3-e158-47e6-9eb6-527377ed3302 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:10:56.970226 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab24b3-e158-47e6-9eb6-527377ed3302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:10:57.042496 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-daab24b3-e158-47e6-9eb6-527377ed3302 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:10:57.043778 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-daab24b3-e158-47e6-9eb6-527377ed3302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:10:57.098979 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:10:57.149144 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 753/1489] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:10:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:10:57.437532 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062f4310-126d-4f3a-817d-1163da679927 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:10:57.439115 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-062f4310-126d-4f3a-817d-1163da679927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:10:57.478682 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:10:57.560465 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062f4310-126d-4f3a-817d-1163da679927 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:10:57.644879 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb1b709-9180-4329-92cd-553a3fa7289d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:10:57.645484 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb1b709-9180-4329-92cd-553a3fa7289d service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:10:57.645781 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb1b709-9180-4329-92cd-553a3fa7289d service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:10:57.646046 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb1b709-9180-4329-92cd-553a3fa7289d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:10:57.647697 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6cb1b709-9180-4329-92cd-553a3fa7289d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:10:57.732032 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:10:57.773238 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb1b709-9180-4329-92cd-553a3fa7289d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:10:57.774688 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb1b709-9180-4329-92cd-553a3fa7289d service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:10:57.793356 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 737/1490] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:10:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:10:59.907854 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deac796-699b-4721-94ea-9c234fa5a603 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:10:59.908482 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deac796-699b-4721-94ea-9c234fa5a603 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:10:59.908482 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deac796-699b-4721-94ea-9c234fa5a603 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:10:59.909801 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 754/1491] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 6 16:10:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:10:59.916218 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5654e4c4-745b-42d1-ac6c-1ebcb27c7345 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:10:59.916996 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5654e4c4-745b-42d1-ac6c-1ebcb27c7345 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:10:59.917159 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5654e4c4-745b-42d1-ac6c-1ebcb27c7345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:10:59.991821 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-5654e4c4-745b-42d1-ac6c-1ebcb27c7345 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:10:59.993088 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5654e4c4-745b-42d1-ac6c-1ebcb27c7345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:11:00.108554 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 738/1492] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 6 16:10:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:11:00.394989 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deac796-699b-4721-94ea-9c234fa5a603 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:11:00.397499 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-4deac796-699b-4721-94ea-9c234fa5a603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:11:00.509873 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deac796-699b-4721-94ea-9c234fa5a603 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:11:00.586134 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593cb860-c503-4fbe-b9dc-a82c88f1785f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:11:00.586641 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593cb860-c503-4fbe-b9dc-a82c88f1785f service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:11:00.586918 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593cb860-c503-4fbe-b9dc-a82c88f1785f service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:11:00.587185 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593cb860-c503-4fbe-b9dc-a82c88f1785f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:11:00.588795 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-593cb860-c503-4fbe-b9dc-a82c88f1785f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:11:00.597741 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:11:00.696436 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593cb860-c503-4fbe-b9dc-a82c88f1785f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:11:00.697890 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593cb860-c503-4fbe-b9dc-a82c88f1785f service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:11:00.716000 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 755/1493] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:11:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:11:10.106465 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=82 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:11:10.713654 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=50 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:12:13.446740 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68942dd-d858-4df5-ab36-eb563a357000 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:12:13.447680 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68942dd-d858-4df5-ab36-eb563a357000 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:12:13.447680 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68942dd-d858-4df5-ab36-eb563a357000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:12:13.532186 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-e68942dd-d858-4df5-ab36-eb563a357000 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:12:13.533342 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e68942dd-d858-4df5-ab36-eb563a357000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:12:13.635985 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 739/1494] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:12:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:12:13.644220 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273eb090-c1c0-437c-9e04-db603874ea0c None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:12:13.644742 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273eb090-c1c0-437c-9e04-db603874ea0c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:12:13.644742 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273eb090-c1c0-437c-9e04-db603874ea0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:12:13.737735 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-273eb090-c1c0-437c-9e04-db603874ea0c None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:12:13.738936 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-273eb090-c1c0-437c-9e04-db603874ea0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:12:13.845116 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 756/1495] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:12:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:12:13.850930 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68942dd-d858-4df5-ab36-eb563a357000 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:12:13.852146 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e68942dd-d858-4df5-ab36-eb563a357000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:12:13.864787 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68942dd-d858-4df5-ab36-eb563a357000 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:12:13.931790 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:12:13.972051 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acf26b1-97d3-4fc9-a241-02028fb34a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:12:13.972663 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acf26b1-97d3-4fc9-a241-02028fb34a1c admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:12:13.972985 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acf26b1-97d3-4fc9-a241-02028fb34a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:12:13.973256 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acf26b1-97d3-4fc9-a241-02028fb34a1c admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:12:13.973908 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acf26b1-97d3-4fc9-a241-02028fb34a1c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:12:13.974207 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acf26b1-97d3-4fc9-a241-02028fb34a1c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:12:13.975275 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: WARNING py.warnings [None req-7acf26b1-97d3-4fc9-a241-02028fb34a1c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 06 16:12:13.975275 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: warnings.warn(msg) Sep 06 16:12:13.975275 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: Sep 06 16:12:13.976206 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acf26b1-97d3-4fc9-a241-02028fb34a1c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:12:13.987803 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 740/1496] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 6 16:12:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:12:13.999836 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273eb090-c1c0-437c-9e04-db603874ea0c None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:12:14.009564 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273eb090-c1c0-437c-9e04-db603874ea0c None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:12:14.079533 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:12:14.115335 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9ae6fb-4847-4201-bd38-5d9d569f9423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:12:14.115908 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9ae6fb-4847-4201-bd38-5d9d569f9423 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:12:14.116189 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9ae6fb-4847-4201-bd38-5d9d569f9423 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:12:14.116455 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9ae6fb-4847-4201-bd38-5d9d569f9423 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:12:14.117096 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9ae6fb-4847-4201-bd38-5d9d569f9423 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:12:14.118816 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: WARNING py.warnings [None req-7d9ae6fb-4847-4201-bd38-5d9d569f9423 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 16:12:14.118816 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: warnings.warn(msg) Sep 06 16:12:14.118816 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: Sep 06 16:12:14.120020 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9ae6fb-4847-4201-bd38-5d9d569f9423 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:12:14.191627 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 757/1497] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:12:13 2022] POST /identity/v3/projects => generated 377 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:12:14.197354 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acf26b1-97d3-4fc9-a241-02028fb34a1c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:12:14.199687 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-7acf26b1-97d3-4fc9-a241-02028fb34a1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:12:14.245377 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acf26b1-97d3-4fc9-a241-02028fb34a1c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:12:14.345440 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554bff45-95f8-4592-94b2-c6503fec9bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:12:14.346057 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554bff45-95f8-4592-94b2-c6503fec9bf0 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:12:14.346327 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554bff45-95f8-4592-94b2-c6503fec9bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:12:14.346575 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554bff45-95f8-4592-94b2-c6503fec9bf0 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:12:14.347220 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554bff45-95f8-4592-94b2-c6503fec9bf0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:12:14.348245 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: WARNING py.warnings [None req-554bff45-95f8-4592-94b2-c6503fec9bf0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 16:12:14.348245 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: warnings.warn(msg) Sep 06 16:12:14.348245 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: Sep 06 16:12:14.349477 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554bff45-95f8-4592-94b2-c6503fec9bf0 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:12:14.394539 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 741/1498] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:12:14 2022] POST /identity/v3/users => generated 289 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:12:14.398838 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9ae6fb-4847-4201-bd38-5d9d569f9423 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:12:14.400468 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7d9ae6fb-4847-4201-bd38-5d9d569f9423 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:12:14.467211 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9ae6fb-4847-4201-bd38-5d9d569f9423 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:12:14.553224 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:12:14.565101 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2d457f-a0bc-4d3c-b427-8fc5a1747f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:12:14.565712 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2d457f-a0bc-4d3c-b427-8fc5a1747f9c admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:12:14.565981 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2d457f-a0bc-4d3c-b427-8fc5a1747f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:12:14.566233 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2d457f-a0bc-4d3c-b427-8fc5a1747f9c admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:12:14.566917 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2d457f-a0bc-4d3c-b427-8fc5a1747f9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:12:14.568285 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: WARNING py.warnings [None req-da2d457f-a0bc-4d3c-b427-8fc5a1747f9c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 06 16:12:14.568285 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: warnings.warn(msg) Sep 06 16:12:14.568285 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: Sep 06 16:12:14.569202 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2d457f-a0bc-4d3c-b427-8fc5a1747f9c admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:12:14.582241 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 758/1499] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:12:14 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:12:14.587113 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554bff45-95f8-4592-94b2-c6503fec9bf0 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:12:14.589231 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-554bff45-95f8-4592-94b2-c6503fec9bf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:12:14.601502 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554bff45-95f8-4592-94b2-c6503fec9bf0 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:12:14.627743 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:12:14.700330 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb72b12f-5056-48de-b2d2-5d5d3be93e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:12:14.700878 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb72b12f-5056-48de-b2d2-5d5d3be93e78 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:12:14.701150 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb72b12f-5056-48de-b2d2-5d5d3be93e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:12:14.701415 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb72b12f-5056-48de-b2d2-5d5d3be93e78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:12:14.702043 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb72b12f-5056-48de-b2d2-5d5d3be93e78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:12:14.703070 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: WARNING py.warnings [None req-eb72b12f-5056-48de-b2d2-5d5d3be93e78 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 06 16:12:14.703070 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: warnings.warn(msg) Sep 06 16:12:14.703070 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: Sep 06 16:12:14.703953 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb72b12f-5056-48de-b2d2-5d5d3be93e78 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:12:14.716539 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 742/1500] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:12:14 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:12:14.721407 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2d457f-a0bc-4d3c-b427-8fc5a1747f9c admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:12:14.723327 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-da2d457f-a0bc-4d3c-b427-8fc5a1747f9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:12:14.765674 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2d457f-a0bc-4d3c-b427-8fc5a1747f9c admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:12:14.863761 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daa3bd9-64cc-4165-9dbb-8a2a171c354b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:12:14.864453 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa3bd9-64cc-4165-9dbb-8a2a171c354b admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:12:14.864731 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa3bd9-64cc-4165-9dbb-8a2a171c354b admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:12:14.865037 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa3bd9-64cc-4165-9dbb-8a2a171c354b admin admin] PATH_INFO: `/v3/projects/cf7f158613aa4bd4994e576779ce4327/users/86c23bad3baf4e808c5a1dcc67c2e40d/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:12:14.889570 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daa3bd9-64cc-4165-9dbb-8a2a171c354b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf7f158613aa4bd4994e576779ce4327, user_id=86c23bad3baf4e808c5a1dcc67c2e40d, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:12:14.890607 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: WARNING py.warnings [None req-4daa3bd9-64cc-4165-9dbb-8a2a171c354b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 16:12:14.890607 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: warnings.warn(msg) Sep 06 16:12:14.890607 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: Sep 06 16:12:14.892539 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daa3bd9-64cc-4165-9dbb-8a2a171c354b admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:12:14.920155 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 759/1501] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:12:14 2022] PUT /identity/v3/projects/cf7f158613aa4bd4994e576779ce4327/users/86c23bad3baf4e808c5a1dcc67c2e40d/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:12:14.932682 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2475b191-540e-4cdb-a71f-f719e18c3921 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:12:14.932999 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2475b191-540e-4cdb-a71f-f719e18c3921 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:12:14.933273 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2475b191-540e-4cdb-a71f-f719e18c3921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:12:14.992515 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-2475b191-540e-4cdb-a71f-f719e18c3921 None None] MFA Rules not processed for user `86c23bad3baf4e808c5a1dcc67c2e40d`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:12:14.993655 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-2475b191-540e-4cdb-a71f-f719e18c3921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:12:15.075721 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 743/1502] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:12:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:12:15.090302 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5927993-102d-41f1-8a6e-b7364b18314e None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:12:15.090719 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5927993-102d-41f1-8a6e-b7364b18314e None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:12:15.091122 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5927993-102d-41f1-8a6e-b7364b18314e None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:12:15.093131 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 760/1503] 10.222.0.43 () {54 vars in 964 bytes} [Tue Sep 6 16:12:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:12:15.098079 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b3306-e754-41d9-883e-abb8124dc6d4 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:12:15.098388 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b3306-e754-41d9-883e-abb8124dc6d4 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:12:15.098638 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b3306-e754-41d9-883e-abb8124dc6d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:12:15.131726 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:12:15.177115 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-727b3306-e754-41d9-883e-abb8124dc6d4 None None] MFA Rules not processed for user `d1b05748e4284153b0728164cd508dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:12:15.178335 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-727b3306-e754-41d9-883e-abb8124dc6d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:12:15.259129 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 744/1504] 10.222.0.43 () {58 vars in 1061 bytes} [Tue Sep 6 16:12:15 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:12:15.267015 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5927993-102d-41f1-8a6e-b7364b18314e None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:12:15.268582 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a5927993-102d-41f1-8a6e-b7364b18314e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:12:15.322386 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:12:15.380670 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5927993-102d-41f1-8a6e-b7364b18314e None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:12:15.464910 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47492bca-5533-45e8-88e7-f8a57eccd8fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:12:15.465575 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47492bca-5533-45e8-88e7-f8a57eccd8fc service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:12:15.465846 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47492bca-5533-45e8-88e7-f8a57eccd8fc service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:12:15.466115 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47492bca-5533-45e8-88e7-f8a57eccd8fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:12:15.467654 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-47492bca-5533-45e8-88e7-f8a57eccd8fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:12:15.569000 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47492bca-5533-45e8-88e7-f8a57eccd8fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:12:15.570438 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47492bca-5533-45e8-88e7-f8a57eccd8fc service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:12:15.573853 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:12:15.588791 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 761/1505] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:12:15 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:12:15.634092 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14899b7d-8378-48f7-9953-406b24047a84 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:12:15.634557 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14899b7d-8378-48f7-9953-406b24047a84 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:12:15.634669 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14899b7d-8378-48f7-9953-406b24047a84 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:12:15.635881 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 745/1506] 10.222.0.43 () {52 vars in 830 bytes} [Tue Sep 6 16:12:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:12:15.639616 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47492bca-5533-45e8-88e7-f8a57eccd8fc service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:12:15.640856 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-47492bca-5533-45e8-88e7-f8a57eccd8fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:12:15.745039 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47492bca-5533-45e8-88e7-f8a57eccd8fc service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:12:15.845373 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2908a06a-211d-4f95-a5c0-d08b3d4f30f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:12:15.845926 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2908a06a-211d-4f95-a5c0-d08b3d4f30f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:12:15.846200 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2908a06a-211d-4f95-a5c0-d08b3d4f30f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:12:15.846554 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2908a06a-211d-4f95-a5c0-d08b3d4f30f9 admin admin] PATH_INFO: `/v3/users/86c23bad3baf4e808c5a1dcc67c2e40d` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:12:15.861225 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2908a06a-211d-4f95-a5c0-d08b3d4f30f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86c23bad3baf4e808c5a1dcc67c2e40d)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:12:15.862856 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: WARNING py.warnings [None req-2908a06a-211d-4f95-a5c0-d08b3d4f30f9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 16:12:15.862856 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: warnings.warn(msg) Sep 06 16:12:15.862856 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: Sep 06 16:12:15.864934 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2908a06a-211d-4f95-a5c0-d08b3d4f30f9 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:12:15.880956 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:12:15.938998 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-2908a06a-211d-4f95-a5c0-d08b3d4f30f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86c23bad3baf4e808c5a1dcc67c2e40d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:12:15.939525 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-2908a06a-211d-4f95-a5c0-d08b3d4f30f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86c23bad3baf4e808c5a1dcc67c2e40d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:12:15.954356 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-2908a06a-211d-4f95-a5c0-d08b3d4f30f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86c23bad3baf4e808c5a1dcc67c2e40d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:12:15.979080 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-2908a06a-211d-4f95-a5c0-d08b3d4f30f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86c23bad3baf4e808c5a1dcc67c2e40d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:12:15.990591 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-2908a06a-211d-4f95-a5c0-d08b3d4f30f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86c23bad3baf4e808c5a1dcc67c2e40d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:12:15.996501 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 762/1507] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:12:15 2022] DELETE /identity/v3/users/86c23bad3baf4e808c5a1dcc67c2e40d => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:12:16.001194 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb2670-c250-439d-89b5-5f2ea22fc3d7 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:12:16.001484 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb2670-c250-439d-89b5-5f2ea22fc3d7 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:12:16.001760 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb2670-c250-439d-89b5-5f2ea22fc3d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:12:16.061217 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-0fbb2670-c250-439d-89b5-5f2ea22fc3d7 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:12:16.062765 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0fbb2670-c250-439d-89b5-5f2ea22fc3d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:12:16.143831 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:12:16.156892 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 746/1508] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:12:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:12:16.166771 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2908a06a-211d-4f95-a5c0-d08b3d4f30f9 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:12:16.168027 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-2908a06a-211d-4f95-a5c0-d08b3d4f30f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:12:16.279378 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2908a06a-211d-4f95-a5c0-d08b3d4f30f9 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:12:16.336725 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:12:16.357920 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe11678-72f2-4a43-85c1-eb8cebb04a77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:12:16.358531 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe11678-72f2-4a43-85c1-eb8cebb04a77 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:12:16.358809 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe11678-72f2-4a43-85c1-eb8cebb04a77 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:12:16.359080 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe11678-72f2-4a43-85c1-eb8cebb04a77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:12:16.360672 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-efe11678-72f2-4a43-85c1-eb8cebb04a77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:12:16.467944 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe11678-72f2-4a43-85c1-eb8cebb04a77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:12:16.469340 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe11678-72f2-4a43-85c1-eb8cebb04a77 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:12:16.488237 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 763/1509] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:12:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:12:16.814371 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbb2670-c250-439d-89b5-5f2ea22fc3d7 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:12:16.815615 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0fbb2670-c250-439d-89b5-5f2ea22fc3d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:12:16.827654 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbb2670-c250-439d-89b5-5f2ea22fc3d7 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:12:16.923395 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b47c78a-6b72-4731-b1ae-e5b0f6e3dcb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:12:16.924026 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b47c78a-6b72-4731-b1ae-e5b0f6e3dcb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:12:16.924294 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b47c78a-6b72-4731-b1ae-e5b0f6e3dcb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:12:16.924554 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b47c78a-6b72-4731-b1ae-e5b0f6e3dcb8 admin admin] PATH_INFO: `/v3/projects/cf7f158613aa4bd4994e576779ce4327` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:12:16.934649 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b47c78a-6b72-4731-b1ae-e5b0f6e3dcb8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf7f158613aa4bd4994e576779ce4327)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:12:16.935691 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: WARNING py.warnings [None req-1b47c78a-6b72-4731-b1ae-e5b0f6e3dcb8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 16:12:16.935691 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: warnings.warn(msg) Sep 06 16:12:16.935691 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: Sep 06 16:12:16.936942 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b47c78a-6b72-4731-b1ae-e5b0f6e3dcb8 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:12:16.996257 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-1b47c78a-6b72-4731-b1ae-e5b0f6e3dcb8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf7f158613aa4bd4994e576779ce4327', 'request_id': 'req-1b47c78a-6b72-4731-b1ae-e5b0f6e3dcb8'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:12:17.002638 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-1b47c78a-6b72-4731-b1ae-e5b0f6e3dcb8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf7f158613aa4bd4994e576779ce4327', 'request_id': 'req-1b47c78a-6b72-4731-b1ae-e5b0f6e3dcb8'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:12:17.013161 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-1b47c78a-6b72-4731-b1ae-e5b0f6e3dcb8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf7f158613aa4bd4994e576779ce4327', 'request_id': 'req-1b47c78a-6b72-4731-b1ae-e5b0f6e3dcb8'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:12:17.043928 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-1b47c78a-6b72-4731-b1ae-e5b0f6e3dcb8 admin admin] The token cache is being invalidate because project cf7f158613aa4bd4994e576779ce4327 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:12:17.044208 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-1b47c78a-6b72-4731-b1ae-e5b0f6e3dcb8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:12:17.044724 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-1b47c78a-6b72-4731-b1ae-e5b0f6e3dcb8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:12:17.046051 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 747/1510] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:12:16 2022] DELETE /identity/v3/projects/cf7f158613aa4bd4994e576779ce4327 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:12:26.485853 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=62 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:12:27.041821 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:13:48.400380 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af95f544-388a-45da-9e16-a38de0290c47 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:13:48.401320 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af95f544-388a-45da-9e16-a38de0290c47 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:13:48.401320 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af95f544-388a-45da-9e16-a38de0290c47 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:13:48.402915 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 764/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 16:13:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:13:48.410262 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41556f05-8217-41dc-9017-a927e5b26762 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:13:48.410730 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41556f05-8217-41dc-9017-a927e5b26762 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:13:48.410836 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41556f05-8217-41dc-9017-a927e5b26762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:13:48.518847 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-41556f05-8217-41dc-9017-a927e5b26762 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:13:48.520743 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-41556f05-8217-41dc-9017-a927e5b26762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:13:48.772949 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 748/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 16:13:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:13:48.933683 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af95f544-388a-45da-9e16-a38de0290c47 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:13:48.936074 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-af95f544-388a-45da-9e16-a38de0290c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:13:49.071518 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af95f544-388a-45da-9e16-a38de0290c47 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:13:49.081836 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa8d028-59d2-4ce8-8f76-e4a2321c232a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63de41f4348243998029f7d07aa5dc40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:13:49.082780 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa8d028-59d2-4ce8-8f76-e4a2321c232a service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:13:49.083104 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa8d028-59d2-4ce8-8f76-e4a2321c232a service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:13:49.083453 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa8d028-59d2-4ce8-8f76-e4a2321c232a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:13:49.094634 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa8d028-59d2-4ce8-8f76-e4a2321c232a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:13:49.096237 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa8d028-59d2-4ce8-8f76-e4a2321c232a service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:13:49.108583 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 765/1513] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:13:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:13:49.947012 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f258e3-9a3b-4bc0-8cb3-dacdf5d676a7 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:13:49.947641 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f258e3-9a3b-4bc0-8cb3-dacdf5d676a7 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:13:49.947641 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f258e3-9a3b-4bc0-8cb3-dacdf5d676a7 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:13:49.949000 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 749/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 16:13:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:13:49.954995 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ebb42f-ed88-40d6-8f2a-47d854160ad8 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:13:49.955705 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ebb42f-ed88-40d6-8f2a-47d854160ad8 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:13:49.955806 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ebb42f-ed88-40d6-8f2a-47d854160ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:13:49.982541 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-33ebb42f-ed88-40d6-8f2a-47d854160ad8 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:13:49.983734 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-33ebb42f-ed88-40d6-8f2a-47d854160ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:13:49.992006 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 766/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 16:13:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:13:50.142179 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f258e3-9a3b-4bc0-8cb3-dacdf5d676a7 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:13:50.158291 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f258e3-9a3b-4bc0-8cb3-dacdf5d676a7 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:13:50.165369 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce82c96-a863-4b26-9a41-6c3c577222d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63de41f4348243998029f7d07aa5dc40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:13:50.165972 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce82c96-a863-4b26-9a41-6c3c577222d5 service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:13:50.166243 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce82c96-a863-4b26-9a41-6c3c577222d5 service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:13:50.166509 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce82c96-a863-4b26-9a41-6c3c577222d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:13:50.175963 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce82c96-a863-4b26-9a41-6c3c577222d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:13:50.177668 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce82c96-a863-4b26-9a41-6c3c577222d5 service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:13:50.190235 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 750/1516] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:13:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:13:50.553024 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbbcab2-19be-47ca-8399-7649930450f4 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:13:50.553311 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbbcab2-19be-47ca-8399-7649930450f4 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:13:50.553589 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbbcab2-19be-47ca-8399-7649930450f4 None None] PATH_INFO: `/v3` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:13:50.554950 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 767/1517] 10.222.0.64 () {54 vars in 925 bytes} [Tue Sep 6 16:13:50 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:13:50.563218 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15abb476-6591-4da9-b28b-ccc6c6cbd1b1 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:13:50.563656 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15abb476-6591-4da9-b28b-ccc6c6cbd1b1 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:13:50.564069 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15abb476-6591-4da9-b28b-ccc6c6cbd1b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:13:50.644032 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-15abb476-6591-4da9-b28b-ccc6c6cbd1b1 None None] MFA Rules not processed for user `71c2912a44f04bd983b4d4112dbee734`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:13:50.645297 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-15abb476-6591-4da9-b28b-ccc6c6cbd1b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:13:50.662901 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:13:50.728068 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 751/1518] 10.222.0.64 () {58 vars in 1015 bytes} [Tue Sep 6 16:13:50 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:13:59.976996 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=56 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:14:00.048317 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a47ab4-a703-4506-b466-5a34b9fd294a None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:00.048547 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a47ab4-a703-4506-b466-5a34b9fd294a None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:00.048901 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a47ab4-a703-4506-b466-5a34b9fd294a None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:00.050436 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 768/1519] 10.222.0.43 () {54 vars in 961 bytes} [Tue Sep 6 16:14:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:14:00.056033 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd6343-b1ec-4a16-babd-889aa927e639 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:00.056608 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd6343-b1ec-4a16-babd-889aa927e639 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:00.056608 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd6343-b1ec-4a16-babd-889aa927e639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:00.078876 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-47fd6343-b1ec-4a16-babd-889aa927e639 None None] MFA Rules not processed for user `71c2912a44f04bd983b4d4112dbee734`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:00.080160 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-47fd6343-b1ec-4a16-babd-889aa927e639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:00.090542 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 752/1520] 10.222.0.43 () {58 vars in 1058 bytes} [Tue Sep 6 16:14:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:14:00.099500 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a47ab4-a703-4506-b466-5a34b9fd294a None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:00.115231 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a47ab4-a703-4506-b466-5a34b9fd294a None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:00.121846 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428f8005-a66e-4f41-bed7-fac8df46db6a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71c2912a44f04bd983b4d4112dbee734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:00.122519 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428f8005-a66e-4f41-bed7-fac8df46db6a service placement] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:00.122789 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428f8005-a66e-4f41-bed7-fac8df46db6a service placement] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:00.123056 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428f8005-a66e-4f41-bed7-fac8df46db6a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:00.133053 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428f8005-a66e-4f41-bed7-fac8df46db6a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:00.134669 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428f8005-a66e-4f41-bed7-fac8df46db6a service placement] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:00.146325 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 769/1521] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:14:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:14:01.434236 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400aff68-048c-4145-8eb4-5724ba5372e2 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:01.435002 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400aff68-048c-4145-8eb4-5724ba5372e2 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:01.435002 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400aff68-048c-4145-8eb4-5724ba5372e2 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:01.436141 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 753/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 16:14:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:14:01.441457 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a22e3-de92-49aa-8bd4-483568b7e955 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:01.441921 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a22e3-de92-49aa-8bd4-483568b7e955 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:01.442031 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a22e3-de92-49aa-8bd4-483568b7e955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:01.463427 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-962a22e3-de92-49aa-8bd4-483568b7e955 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:01.464963 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-962a22e3-de92-49aa-8bd4-483568b7e955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:01.473797 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 770/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 16:14:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:14:01.628872 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400aff68-048c-4145-8eb4-5724ba5372e2 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:01.647529 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400aff68-048c-4145-8eb4-5724ba5372e2 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:01.655266 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c46997-42d4-4921-9858-377f3bfa1518 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63de41f4348243998029f7d07aa5dc40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:01.656096 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c46997-42d4-4921-9858-377f3bfa1518 service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:01.656471 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c46997-42d4-4921-9858-377f3bfa1518 service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:01.656891 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c46997-42d4-4921-9858-377f3bfa1518 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:01.669169 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c46997-42d4-4921-9858-377f3bfa1518 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:01.673462 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c46997-42d4-4921-9858-377f3bfa1518 service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:01.688098 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 754/1524] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:14:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:14:02.567570 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a540b687-fe77-440b-9fc5-1c672be129b0 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:02.568155 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a540b687-fe77-440b-9fc5-1c672be129b0 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:02.568155 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a540b687-fe77-440b-9fc5-1c672be129b0 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:02.569540 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 771/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 16:14:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:14:02.574938 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adea88bb-370f-44d5-8bdc-712675ec588a None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:02.575389 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adea88bb-370f-44d5-8bdc-712675ec588a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:02.575503 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adea88bb-370f-44d5-8bdc-712675ec588a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:02.596908 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-adea88bb-370f-44d5-8bdc-712675ec588a None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:02.598163 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-adea88bb-370f-44d5-8bdc-712675ec588a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:02.607792 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 755/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 16:14:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:14:02.769989 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a540b687-fe77-440b-9fc5-1c672be129b0 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:02.791174 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a540b687-fe77-440b-9fc5-1c672be129b0 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:02.797776 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f909250b-4381-4b53-ad6e-b1585e5e3538 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63de41f4348243998029f7d07aa5dc40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:02.798394 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f909250b-4381-4b53-ad6e-b1585e5e3538 service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:02.798671 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f909250b-4381-4b53-ad6e-b1585e5e3538 service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:02.798934 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f909250b-4381-4b53-ad6e-b1585e5e3538 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:02.808479 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f909250b-4381-4b53-ad6e-b1585e5e3538 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:02.810239 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f909250b-4381-4b53-ad6e-b1585e5e3538 service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:02.820813 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 772/1527] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:14:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:14:06.530326 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93007986-9ad0-43b1-99a8-d3f824fde272 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:06.530889 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93007986-9ad0-43b1-99a8-d3f824fde272 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:06.530889 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93007986-9ad0-43b1-99a8-d3f824fde272 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:06.532190 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 756/1528] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 16:14:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:14:06.537655 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc260f5-aee5-4e09-9c71-d7804c3f31f1 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:06.538113 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc260f5-aee5-4e09-9c71-d7804c3f31f1 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:06.538219 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc260f5-aee5-4e09-9c71-d7804c3f31f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:06.569560 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-9bc260f5-aee5-4e09-9c71-d7804c3f31f1 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:06.571097 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-9bc260f5-aee5-4e09-9c71-d7804c3f31f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:06.580538 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 773/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 16:14:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:14:06.868026 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93007986-9ad0-43b1-99a8-d3f824fde272 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:06.869792 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-93007986-9ad0-43b1-99a8-d3f824fde272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:06.969186 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93007986-9ad0-43b1-99a8-d3f824fde272 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:06.973109 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cee13f7-64b3-4f2d-98a9-8458744cdfe7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:06.973720 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cee13f7-64b3-4f2d-98a9-8458744cdfe7 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:06.973997 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cee13f7-64b3-4f2d-98a9-8458744cdfe7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:06.974268 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cee13f7-64b3-4f2d-98a9-8458744cdfe7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:06.984225 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cee13f7-64b3-4f2d-98a9-8458744cdfe7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:06.985680 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cee13f7-64b3-4f2d-98a9-8458744cdfe7 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:06.995874 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 757/1530] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:14:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:14:08.907539 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afc9e10-7983-49e5-95a3-973f76c06eff None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:08.908174 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afc9e10-7983-49e5-95a3-973f76c06eff None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:08.908419 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afc9e10-7983-49e5-95a3-973f76c06eff None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:08.910098 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 774/1531] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 16:14:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:14:08.917081 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31549826-88fa-4e5f-927a-eae766d837bf None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:08.917617 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31549826-88fa-4e5f-927a-eae766d837bf None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:08.917726 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31549826-88fa-4e5f-927a-eae766d837bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:08.942475 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-31549826-88fa-4e5f-927a-eae766d837bf None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:08.944092 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-31549826-88fa-4e5f-927a-eae766d837bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:08.953697 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 758/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 16:14:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:14:09.238254 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afc9e10-7983-49e5-95a3-973f76c06eff None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:09.240100 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-1afc9e10-7983-49e5-95a3-973f76c06eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:09.253720 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afc9e10-7983-49e5-95a3-973f76c06eff None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:09.257539 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5ea63b-9543-4303-8112-01be267438e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:09.258147 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5ea63b-9543-4303-8112-01be267438e8 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:09.258466 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5ea63b-9543-4303-8112-01be267438e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:09.258736 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5ea63b-9543-4303-8112-01be267438e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:09.268858 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5ea63b-9543-4303-8112-01be267438e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:09.270350 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5ea63b-9543-4303-8112-01be267438e8 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:09.281415 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 775/1533] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:14:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:14:12.937361 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d0ea3d-4898-4e18-83e7-c7806d995274 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:12.938226 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d0ea3d-4898-4e18-83e7-c7806d995274 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:12.938226 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d0ea3d-4898-4e18-83e7-c7806d995274 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:12.940273 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 759/1534] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 16:14:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:14:12.947986 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbca20e9-c352-46ba-b417-579cc171c31a None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:12.948456 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbca20e9-c352-46ba-b417-579cc171c31a None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:12.948552 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbca20e9-c352-46ba-b417-579cc171c31a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:12.973245 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-bbca20e9-c352-46ba-b417-579cc171c31a None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:12.974439 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-bbca20e9-c352-46ba-b417-579cc171c31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:12.983814 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 776/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 16:14:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:14:13.532275 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d0ea3d-4898-4e18-83e7-c7806d995274 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:13.534589 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-44d0ea3d-4898-4e18-83e7-c7806d995274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:13.621632 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:14:13.637046 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d0ea3d-4898-4e18-83e7-c7806d995274 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:13.641441 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe6dafe-30b1-4ecd-b58b-b22bb3751f3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:13.642047 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe6dafe-30b1-4ecd-b58b-b22bb3751f3d service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:13.642319 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe6dafe-30b1-4ecd-b58b-b22bb3751f3d service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:13.642583 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe6dafe-30b1-4ecd-b58b-b22bb3751f3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:13.652603 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe6dafe-30b1-4ecd-b58b-b22bb3751f3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:13.654089 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe6dafe-30b1-4ecd-b58b-b22bb3751f3d service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:13.663926 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 760/1536] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:14:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:14:15.625985 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82776625-32c3-46d5-ba8d-cac752c77f96 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:15.626520 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82776625-32c3-46d5-ba8d-cac752c77f96 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:15.626520 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82776625-32c3-46d5-ba8d-cac752c77f96 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:15.627812 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 777/1537] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 16:14:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:14:15.633787 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ede002d-bf1c-4004-bcdb-bc049b777831 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:15.634304 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ede002d-bf1c-4004-bcdb-bc049b777831 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:15.634304 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ede002d-bf1c-4004-bcdb-bc049b777831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:15.654537 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-4ede002d-bf1c-4004-bcdb-bc049b777831 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:15.655733 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-4ede002d-bf1c-4004-bcdb-bc049b777831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:15.663887 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 761/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 16:14:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:14:16.232230 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82776625-32c3-46d5-ba8d-cac752c77f96 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:16.234625 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-82776625-32c3-46d5-ba8d-cac752c77f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:16.252036 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82776625-32c3-46d5-ba8d-cac752c77f96 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:16.259384 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce635dc0-a33f-4e0a-a8e0-c06eadf0aeb3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:16.260198 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce635dc0-a33f-4e0a-a8e0-c06eadf0aeb3 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:16.260567 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce635dc0-a33f-4e0a-a8e0-c06eadf0aeb3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:16.260862 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce635dc0-a33f-4e0a-a8e0-c06eadf0aeb3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:16.270925 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce635dc0-a33f-4e0a-a8e0-c06eadf0aeb3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:16.272405 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce635dc0-a33f-4e0a-a8e0-c06eadf0aeb3 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:16.282894 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 778/1539] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:14:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:14:25.648861 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=11 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:14:26.278272 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=33 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:41.431430 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476db007-10a5-4a5f-ab6d-903f5b0b97b1 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:41.432569 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476db007-10a5-4a5f-ab6d-903f5b0b97b1 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:41.433181 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476db007-10a5-4a5f-ab6d-903f5b0b97b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:41.459631 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-476db007-10a5-4a5f-ab6d-903f5b0b97b1 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:41.461384 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-476db007-10a5-4a5f-ab6d-903f5b0b97b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:41.475587 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 762/1540] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:15:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:15:41.485210 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83918d02-2453-4d94-9702-e62d846a8d47 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:41.486292 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83918d02-2453-4d94-9702-e62d846a8d47 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:41.486799 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83918d02-2453-4d94-9702-e62d846a8d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:41.512446 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-83918d02-2453-4d94-9702-e62d846a8d47 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:41.514251 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-83918d02-2453-4d94-9702-e62d846a8d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:41.526973 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 779/1541] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:15:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:41.532464 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476db007-10a5-4a5f-ab6d-903f5b0b97b1 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:41.542088 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207eeb7c-ab99-45ac-ba4f-7b36e86d23b0 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:41.542945 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207eeb7c-ab99-45ac-ba4f-7b36e86d23b0 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:41.545558 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476db007-10a5-4a5f-ab6d-903f5b0b97b1 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:41.547063 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207eeb7c-ab99-45ac-ba4f-7b36e86d23b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:41.556103 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0077ebea-9bb1-4e26-95ff-fecd8e13759c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:41.561440 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0077ebea-9bb1-4e26-95ff-fecd8e13759c admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:41.561802 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0077ebea-9bb1-4e26-95ff-fecd8e13759c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:41.561975 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0077ebea-9bb1-4e26-95ff-fecd8e13759c admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:41.562672 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0077ebea-9bb1-4e26-95ff-fecd8e13759c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:15:41.563048 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0077ebea-9bb1-4e26-95ff-fecd8e13759c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:41.565405 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0077ebea-9bb1-4e26-95ff-fecd8e13759c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:41.587240 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-207eeb7c-ab99-45ac-ba4f-7b36e86d23b0 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:41.588856 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-207eeb7c-ab99-45ac-ba4f-7b36e86d23b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:41.590170 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 763/1542] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 6 16:15:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:15:41.594339 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88cb31a-aaeb-4442-8fdc-0e3ac34f695f None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:41.594339 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88cb31a-aaeb-4442-8fdc-0e3ac34f695f None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:41.594339 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88cb31a-aaeb-4442-8fdc-0e3ac34f695f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:41.608011 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 780/1543] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:15:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:15:41.609259 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207eeb7c-ab99-45ac-ba4f-7b36e86d23b0 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:41.626706 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207eeb7c-ab99-45ac-ba4f-7b36e86d23b0 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:41.632123 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-a88cb31a-aaeb-4442-8fdc-0e3ac34f695f None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:41.633423 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a88cb31a-aaeb-4442-8fdc-0e3ac34f695f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:41.646728 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af23f8c-bc5c-4599-adc4-dc70a6fc5896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:41.648366 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af23f8c-bc5c-4599-adc4-dc70a6fc5896 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:41.650229 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af23f8c-bc5c-4599-adc4-dc70a6fc5896 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:41.650755 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af23f8c-bc5c-4599-adc4-dc70a6fc5896 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:41.651630 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af23f8c-bc5c-4599-adc4-dc70a6fc5896 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:41.654207 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af23f8c-bc5c-4599-adc4-dc70a6fc5896 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:41.661602 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 764/1544] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:15:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:41.665638 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceee40c-6ee6-449d-a59c-f789232ca58b None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:41.666279 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceee40c-6ee6-449d-a59c-f789232ca58b None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:41.666942 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceee40c-6ee6-449d-a59c-f789232ca58b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:41.702013 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-eceee40c-6ee6-449d-a59c-f789232ca58b None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:41.708345 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-eceee40c-6ee6-449d-a59c-f789232ca58b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:41.729591 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 765/1545] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:15:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:41.731228 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4266c8c7-7f46-4f6c-b766-f2ba91bdcde7 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:41.731606 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4266c8c7-7f46-4f6c-b766-f2ba91bdcde7 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:41.732021 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4266c8c7-7f46-4f6c-b766-f2ba91bdcde7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:41.754909 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 781/1546] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:15:41 2022] POST /identity/v3/projects => generated 377 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:15:41.758935 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17db9554-8a20-4711-89f3-28313cdf79b0 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:41.758935 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17db9554-8a20-4711-89f3-28313cdf79b0 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:41.758935 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17db9554-8a20-4711-89f3-28313cdf79b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:41.778852 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-4266c8c7-7f46-4f6c-b766-f2ba91bdcde7 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:41.786516 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-4266c8c7-7f46-4f6c-b766-f2ba91bdcde7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:41.803721 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-17db9554-8a20-4711-89f3-28313cdf79b0 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:41.805196 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-17db9554-8a20-4711-89f3-28313cdf79b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:41.882897 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 766/1547] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:15:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:41.884638 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16de5c8a-2ce8-4c95-b9c3-10c2bc50649f None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:41.885182 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16de5c8a-2ce8-4c95-b9c3-10c2bc50649f None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:41.885804 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16de5c8a-2ce8-4c95-b9c3-10c2bc50649f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:41.906921 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-16de5c8a-2ce8-4c95-b9c3-10c2bc50649f None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:41.908198 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-16de5c8a-2ce8-4c95-b9c3-10c2bc50649f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:41.914458 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 782/1548] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:15:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:41.916552 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eac5c9-080a-4e66-9381-680363b6c1aa None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:41.917041 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eac5c9-080a-4e66-9381-680363b6c1aa None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:41.917501 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eac5c9-080a-4e66-9381-680363b6c1aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:41.920870 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 767/1549] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:15:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:41.921724 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16de5c8a-2ce8-4c95-b9c3-10c2bc50649f None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:41.931911 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16de5c8a-2ce8-4c95-b9c3-10c2bc50649f None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:41.937321 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacc2853-e0dd-49f6-805c-31d05432da4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:41.937943 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacc2853-e0dd-49f6-805c-31d05432da4b admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:41.938346 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacc2853-e0dd-49f6-805c-31d05432da4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:41.941117 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacc2853-e0dd-49f6-805c-31d05432da4b admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:41.942112 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacc2853-e0dd-49f6-805c-31d05432da4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:41.942587 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-74eac5c9-080a-4e66-9381-680363b6c1aa None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:41.943788 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-74eac5c9-080a-4e66-9381-680363b6c1aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:41.944726 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacc2853-e0dd-49f6-805c-31d05432da4b admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:41.955648 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 783/1550] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:15:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:41.955648 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23980c35-88cc-45de-8377-474c900d800d None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:41.955648 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23980c35-88cc-45de-8377-474c900d800d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:41.955648 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23980c35-88cc-45de-8377-474c900d800d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:41.975880 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 768/1551] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:15:41 2022] POST /identity/v3/users => generated 289 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:15:41.977691 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d4a1d5-1778-4db5-b0ec-ccddecf2fbef None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:41.977913 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d4a1d5-1778-4db5-b0ec-ccddecf2fbef None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:41.978181 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d4a1d5-1778-4db5-b0ec-ccddecf2fbef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:41.984925 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-23980c35-88cc-45de-8377-474c900d800d None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:41.986132 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-23980c35-88cc-45de-8377-474c900d800d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:41.994630 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 784/1552] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:15:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:41.996082 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a10ed35-01cd-4ba7-afcc-e2609bcbfb4b None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:41.996362 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a10ed35-01cd-4ba7-afcc-e2609bcbfb4b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:41.996629 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a10ed35-01cd-4ba7-afcc-e2609bcbfb4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:42.005795 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-b0d4a1d5-1778-4db5-b0ec-ccddecf2fbef None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:42.007159 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b0d4a1d5-1778-4db5-b0ec-ccddecf2fbef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:42.016936 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 769/1553] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:15:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:42.017366 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-4a10ed35-01cd-4ba7-afcc-e2609bcbfb4b None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:42.018108 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d4a1d5-1778-4db5-b0ec-ccddecf2fbef None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:42.018689 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-4a10ed35-01cd-4ba7-afcc-e2609bcbfb4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:42.029819 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 785/1554] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:15:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:42.031150 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a10ed35-01cd-4ba7-afcc-e2609bcbfb4b None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:42.037139 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d4a1d5-1778-4db5-b0ec-ccddecf2fbef None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:42.042113 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a10ed35-01cd-4ba7-afcc-e2609bcbfb4b None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:42.042351 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5003d4de-e3d2-4435-8cd4-26168ae13947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:42.042953 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5003d4de-e3d2-4435-8cd4-26168ae13947 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:42.043232 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5003d4de-e3d2-4435-8cd4-26168ae13947 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:42.043600 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5003d4de-e3d2-4435-8cd4-26168ae13947 admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:42.044397 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5003d4de-e3d2-4435-8cd4-26168ae13947 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:15:42.044832 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5003d4de-e3d2-4435-8cd4-26168ae13947 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:42.047321 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5003d4de-e3d2-4435-8cd4-26168ae13947 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:42.047724 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c0af56-dd9e-4c17-aff9-e046b3bb7245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:42.048444 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c0af56-dd9e-4c17-aff9-e046b3bb7245 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:42.048819 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c0af56-dd9e-4c17-aff9-e046b3bb7245 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:42.049097 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c0af56-dd9e-4c17-aff9-e046b3bb7245 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:42.049745 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c0af56-dd9e-4c17-aff9-e046b3bb7245 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:15:42.050541 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c0af56-dd9e-4c17-aff9-e046b3bb7245 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:42.051094 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: WARNING py.warnings [None req-79c0af56-dd9e-4c17-aff9-e046b3bb7245 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 06 16:15:42.051094 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: warnings.warn(msg) Sep 06 16:15:42.051094 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: Sep 06 16:15:42.052117 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c0af56-dd9e-4c17-aff9-e046b3bb7245 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:42.066000 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 786/1555] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 6 16:15:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:15:42.068360 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a65706f-1f09-4a48-a995-6f1e1e880b12 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:42.068360 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a65706f-1f09-4a48-a995-6f1e1e880b12 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:42.068601 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 770/1556] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 6 16:15:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:15:42.069333 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5003d4de-e3d2-4435-8cd4-26168ae13947 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:42.073608 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a65706f-1f09-4a48-a995-6f1e1e880b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:42.086950 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5003d4de-e3d2-4435-8cd4-26168ae13947 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:42.093185 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df156ae-7359-4e08-992f-dfcd1772fb1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:42.093877 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df156ae-7359-4e08-992f-dfcd1772fb1d admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:42.094247 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df156ae-7359-4e08-992f-dfcd1772fb1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:42.094742 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df156ae-7359-4e08-992f-dfcd1772fb1d admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:42.095539 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df156ae-7359-4e08-992f-dfcd1772fb1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:15:42.095899 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df156ae-7359-4e08-992f-dfcd1772fb1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:42.097832 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df156ae-7359-4e08-992f-dfcd1772fb1d admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:42.101873 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-9a65706f-1f09-4a48-a995-6f1e1e880b12 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:42.103437 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-9a65706f-1f09-4a48-a995-6f1e1e880b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:42.118873 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 771/1558] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 6 16:15:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:15:42.119152 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 787/1558] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:15:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:42.119330 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df156ae-7359-4e08-992f-dfcd1772fb1d admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:42.119467 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d583aa-b304-4757-b4f4-1e01767b277d None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:42.119467 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d583aa-b304-4757-b4f4-1e01767b277d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:42.119467 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d583aa-b304-4757-b4f4-1e01767b277d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:42.131167 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df156ae-7359-4e08-992f-dfcd1772fb1d admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:42.135979 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9820be-fbd8-4797-8da2-e3c36797fc66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:42.136592 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9820be-fbd8-4797-8da2-e3c36797fc66 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:42.136900 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9820be-fbd8-4797-8da2-e3c36797fc66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:42.137168 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9820be-fbd8-4797-8da2-e3c36797fc66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:42.137941 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9820be-fbd8-4797-8da2-e3c36797fc66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:42.139615 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9820be-fbd8-4797-8da2-e3c36797fc66 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:42.139727 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-84d583aa-b304-4757-b4f4-1e01767b277d None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:42.140887 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-84d583aa-b304-4757-b4f4-1e01767b277d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:42.148894 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 788/1559] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:15:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:42.150176 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f6df3c-238c-4f64-ab63-7f5d46683d8e None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:42.150461 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f6df3c-238c-4f64-ab63-7f5d46683d8e None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:42.150627 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 772/1560] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:42 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:15:42.150743 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f6df3c-238c-4f64-ab63-7f5d46683d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:42.152048 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a848e5-d6bf-4857-ae3a-fd93db5254da None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:42.152329 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a848e5-d6bf-4857-ae3a-fd93db5254da None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:42.152674 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a848e5-d6bf-4857-ae3a-fd93db5254da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:42.173373 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-d4a848e5-d6bf-4857-ae3a-fd93db5254da None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:42.173590 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-a4f6df3c-238c-4f64-ab63-7f5d46683d8e None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:42.174474 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d4a848e5-d6bf-4857-ae3a-fd93db5254da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:42.174686 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a4f6df3c-238c-4f64-ab63-7f5d46683d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:42.182958 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 789/1561] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:15:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:42.183954 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f6df3c-238c-4f64-ab63-7f5d46683d8e None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:42.184443 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 773/1562] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:15:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:42.185275 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a848e5-d6bf-4857-ae3a-fd93db5254da None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:42.196838 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a848e5-d6bf-4857-ae3a-fd93db5254da None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:42.197999 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f6df3c-238c-4f64-ab63-7f5d46683d8e None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:42.201942 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c181b53-a3d4-4dcc-80b3-3eb323e5d423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:42.202576 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c181b53-a3d4-4dcc-80b3-3eb323e5d423 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:42.202874 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c181b53-a3d4-4dcc-80b3-3eb323e5d423 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:42.203109 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c181b53-a3d4-4dcc-80b3-3eb323e5d423 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:42.203224 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e647b68-610e-45dc-8b87-5fdf328bce0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:42.203734 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c181b53-a3d4-4dcc-80b3-3eb323e5d423 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:42.203853 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e647b68-610e-45dc-8b87-5fdf328bce0a admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:42.204089 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e647b68-610e-45dc-8b87-5fdf328bce0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:42.204371 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e647b68-610e-45dc-8b87-5fdf328bce0a admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:42.204786 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: WARNING py.warnings [None req-8c181b53-a3d4-4dcc-80b3-3eb323e5d423 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 16:15:42.204786 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: warnings.warn(msg) Sep 06 16:15:42.204786 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: Sep 06 16:15:42.205059 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e647b68-610e-45dc-8b87-5fdf328bce0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:42.205968 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c181b53-a3d4-4dcc-80b3-3eb323e5d423 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:42.206914 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e647b68-610e-45dc-8b87-5fdf328bce0a admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:42.240593 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 774/1563] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:15:42 2022] POST /identity/v3/projects => generated 391 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:15:42.240593 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c181b53-a3d4-4dcc-80b3-3eb323e5d423 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:42.245430 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 790/1564] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:15:42 2022] POST /identity/v3/projects => generated 387 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:15:42.245430 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e647b68-610e-45dc-8b87-5fdf328bce0a admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:42.278497 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:42.296675 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:42.314793 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c181b53-a3d4-4dcc-80b3-3eb323e5d423 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:42.316181 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e647b68-610e-45dc-8b87-5fdf328bce0a admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:42.319408 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb169d1-cb49-4d40-8db6-dedfc1e528bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:42.320018 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb169d1-cb49-4d40-8db6-dedfc1e528bb admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:42.320319 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb169d1-cb49-4d40-8db6-dedfc1e528bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:42.320586 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb169d1-cb49-4d40-8db6-dedfc1e528bb admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:42.321321 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb169d1-cb49-4d40-8db6-dedfc1e528bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:15:42.321485 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5acd90-89ff-4b99-b645-8ab7983613a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:42.321643 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb169d1-cb49-4d40-8db6-dedfc1e528bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:42.322103 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5acd90-89ff-4b99-b645-8ab7983613a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:42.322454 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5acd90-89ff-4b99-b645-8ab7983613a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:42.322742 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5acd90-89ff-4b99-b645-8ab7983613a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:42.323258 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb169d1-cb49-4d40-8db6-dedfc1e528bb admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:42.323422 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5acd90-89ff-4b99-b645-8ab7983613a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:42.325366 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5acd90-89ff-4b99-b645-8ab7983613a0 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:42.343728 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 775/1565] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 6 16:15:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:15:42.344913 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb169d1-cb49-4d40-8db6-dedfc1e528bb admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:42.358507 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 791/1566] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:15:42 2022] POST /identity/v3/projects => generated 405 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:15:42.358730 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb169d1-cb49-4d40-8db6-dedfc1e528bb admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:42.358847 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5acd90-89ff-4b99-b645-8ab7983613a0 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:42.424241 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5acd90-89ff-4b99-b645-8ab7983613a0 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:42.428817 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dda7521-4179-4b1a-afc0-f91b13c814dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:42.429456 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dda7521-4179-4b1a-afc0-f91b13c814dd admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:42.429611 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126d29ee-b5f8-484c-971f-98d130aec6a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:42.429791 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dda7521-4179-4b1a-afc0-f91b13c814dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:42.430063 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dda7521-4179-4b1a-afc0-f91b13c814dd admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:42.430380 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126d29ee-b5f8-484c-971f-98d130aec6a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:42.430712 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126d29ee-b5f8-484c-971f-98d130aec6a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:42.430809 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dda7521-4179-4b1a-afc0-f91b13c814dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:15:42.430980 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126d29ee-b5f8-484c-971f-98d130aec6a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:42.431098 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dda7521-4179-4b1a-afc0-f91b13c814dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:42.431779 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126d29ee-b5f8-484c-971f-98d130aec6a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:42.432968 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dda7521-4179-4b1a-afc0-f91b13c814dd admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:42.433490 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126d29ee-b5f8-484c-971f-98d130aec6a0 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:42.447995 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 792/1567] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:42 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:15:42.447995 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126d29ee-b5f8-484c-971f-98d130aec6a0 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:42.448386 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 776/1568] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 6 16:15:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:15:42.448386 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dda7521-4179-4b1a-afc0-f91b13c814dd admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:42.458859 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126d29ee-b5f8-484c-971f-98d130aec6a0 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:42.463851 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dda7521-4179-4b1a-afc0-f91b13c814dd admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:42.464376 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e288e363-dcc5-45ff-85b1-02dc6e3bbb30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:42.465056 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e288e363-dcc5-45ff-85b1-02dc6e3bbb30 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:42.465358 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e288e363-dcc5-45ff-85b1-02dc6e3bbb30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:42.465608 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e288e363-dcc5-45ff-85b1-02dc6e3bbb30 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:42.466384 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e288e363-dcc5-45ff-85b1-02dc6e3bbb30 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:15:42.466565 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e288e363-dcc5-45ff-85b1-02dc6e3bbb30 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:42.468196 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e288e363-dcc5-45ff-85b1-02dc6e3bbb30 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:42.469163 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026c6657-9657-468e-8c80-4b7c1138320a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:42.469986 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026c6657-9657-468e-8c80-4b7c1138320a admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:42.470113 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026c6657-9657-468e-8c80-4b7c1138320a admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:42.470304 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026c6657-9657-468e-8c80-4b7c1138320a admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:42.470959 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026c6657-9657-468e-8c80-4b7c1138320a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:15:42.471253 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026c6657-9657-468e-8c80-4b7c1138320a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:42.472956 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026c6657-9657-468e-8c80-4b7c1138320a admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:42.481410 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 793/1569] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 6 16:15:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:15:42.484027 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e288e363-dcc5-45ff-85b1-02dc6e3bbb30 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:42.494491 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 777/1570] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 6 16:15:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:15:42.494491 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026c6657-9657-468e-8c80-4b7c1138320a admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:42.495524 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e288e363-dcc5-45ff-85b1-02dc6e3bbb30 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:42.501097 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd164d1-7385-4b1e-9a49-9d1364d2ec37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:42.501727 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd164d1-7385-4b1e-9a49-9d1364d2ec37 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:42.502000 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd164d1-7385-4b1e-9a49-9d1364d2ec37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:42.502263 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd164d1-7385-4b1e-9a49-9d1364d2ec37 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:42.502904 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd164d1-7385-4b1e-9a49-9d1364d2ec37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:42.503980 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: WARNING py.warnings [None req-fcd164d1-7385-4b1e-9a49-9d1364d2ec37 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 16:15:42.503980 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: warnings.warn(msg) Sep 06 16:15:42.503980 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: Sep 06 16:15:42.505021 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026c6657-9657-468e-8c80-4b7c1138320a admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:42.505298 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd164d1-7385-4b1e-9a49-9d1364d2ec37 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:42.509970 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa8cf58-a757-4a34-b5da-dab38740d895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:42.510588 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8cf58-a757-4a34-b5da-dab38740d895 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:42.510931 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8cf58-a757-4a34-b5da-dab38740d895 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:42.511262 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8cf58-a757-4a34-b5da-dab38740d895 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:42.511895 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa8cf58-a757-4a34-b5da-dab38740d895 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:42.514085 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa8cf58-a757-4a34-b5da-dab38740d895 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:42.531864 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 794/1571] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:15:42 2022] POST /identity/v3/users => generated 296 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:15:42.532887 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd164d1-7385-4b1e-9a49-9d1364d2ec37 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:42.544362 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd164d1-7385-4b1e-9a49-9d1364d2ec37 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:42.546443 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 778/1572] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:15:42 2022] POST /identity/v3/users => generated 294 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:15:42.547126 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa8cf58-a757-4a34-b5da-dab38740d895 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:42.555173 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10889fb5-9062-4213-beb9-67b5a30d58b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:42.556543 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10889fb5-9062-4213-beb9-67b5a30d58b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:42.557199 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10889fb5-9062-4213-beb9-67b5a30d58b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:42.557774 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10889fb5-9062-4213-beb9-67b5a30d58b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:42.557924 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa8cf58-a757-4a34-b5da-dab38740d895 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:42.558980 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10889fb5-9062-4213-beb9-67b5a30d58b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:42.562464 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10889fb5-9062-4213-beb9-67b5a30d58b6 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:42.563309 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34e8509-0179-4b9b-99cf-a0dba0bbdc2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:42.564279 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34e8509-0179-4b9b-99cf-a0dba0bbdc2f admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:42.564550 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34e8509-0179-4b9b-99cf-a0dba0bbdc2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:42.564844 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34e8509-0179-4b9b-99cf-a0dba0bbdc2f admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:42.565477 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34e8509-0179-4b9b-99cf-a0dba0bbdc2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:42.567446 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34e8509-0179-4b9b-99cf-a0dba0bbdc2f admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:42.591923 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 795/1573] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:15:42 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:15:42.592373 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 779/1574] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:15:42 2022] POST /identity/v3/users => generated 303 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:15:42.593029 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10889fb5-9062-4213-beb9-67b5a30d58b6 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:42.593533 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34e8509-0179-4b9b-99cf-a0dba0bbdc2f admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:42.606416 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34e8509-0179-4b9b-99cf-a0dba0bbdc2f admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:42.661337 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10889fb5-9062-4213-beb9-67b5a30d58b6 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:42.665481 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be00546-afc0-43c6-a2f9-cfb275468ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:42.666068 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd35ffd-891e-4af4-af5c-164720bffd52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:42.666202 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be00546-afc0-43c6-a2f9-cfb275468ddf admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:42.666348 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be00546-afc0-43c6-a2f9-cfb275468ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:42.666611 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be00546-afc0-43c6-a2f9-cfb275468ddf admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:42.666720 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd35ffd-891e-4af4-af5c-164720bffd52 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:42.666998 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd35ffd-891e-4af4-af5c-164720bffd52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:42.667241 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be00546-afc0-43c6-a2f9-cfb275468ddf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:42.667344 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd35ffd-891e-4af4-af5c-164720bffd52 admin admin] PATH_INFO: `/v3/projects/ad0bef00a05c4029a009992a7f96fa80/users/e225aec64aee47449542aecbec0ae561/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:42.669182 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be00546-afc0-43c6-a2f9-cfb275468ddf admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:42.682475 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd35ffd-891e-4af4-af5c-164720bffd52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad0bef00a05c4029a009992a7f96fa80, user_id=e225aec64aee47449542aecbec0ae561, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:42.685027 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:42.685283 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd35ffd-891e-4af4-af5c-164720bffd52 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:42.688087 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:42.701659 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 780/1575] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:15:42 2022] POST /identity/v3/projects => generated 375 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:15:42.704351 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 796/1576] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:15:42 2022] PUT /identity/v3/projects/ad0bef00a05c4029a009992a7f96fa80/users/e225aec64aee47449542aecbec0ae561/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:42.705707 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be00546-afc0-43c6-a2f9-cfb275468ddf admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:42.706232 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd35ffd-891e-4af4-af5c-164720bffd52 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:42.770293 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd35ffd-891e-4af4-af5c-164720bffd52 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:42.773740 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be00546-afc0-43c6-a2f9-cfb275468ddf admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:42.775119 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b301918f-deb0-4556-b97d-a60e611d1957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:42.775733 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b301918f-deb0-4556-b97d-a60e611d1957 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:42.776007 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b301918f-deb0-4556-b97d-a60e611d1957 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:42.776277 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b301918f-deb0-4556-b97d-a60e611d1957 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:42.776963 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b301918f-deb0-4556-b97d-a60e611d1957 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:42.778780 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c2c7b6-506e-4902-a334-12dd1f71e627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:42.778904 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b301918f-deb0-4556-b97d-a60e611d1957 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:42.779418 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c2c7b6-506e-4902-a334-12dd1f71e627 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:42.779769 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c2c7b6-506e-4902-a334-12dd1f71e627 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:42.780005 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c2c7b6-506e-4902-a334-12dd1f71e627 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:42.780644 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c2c7b6-506e-4902-a334-12dd1f71e627 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:42.782615 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c2c7b6-506e-4902-a334-12dd1f71e627 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:42.809085 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 797/1577] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:15:42 2022] POST /identity/v3/projects => generated 373 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:15:42.809735 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b301918f-deb0-4556-b97d-a60e611d1957 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:42.817694 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 781/1578] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:15:42 2022] POST /identity/v3/projects => generated 377 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:15:42.821874 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c2c7b6-506e-4902-a334-12dd1f71e627 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:42.878641 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b301918f-deb0-4556-b97d-a60e611d1957 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:42.894525 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c2c7b6-506e-4902-a334-12dd1f71e627 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:42.899833 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9ffa85-d0bd-4959-a214-cc4a7f1e530e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:42.900448 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9ffa85-d0bd-4959-a214-cc4a7f1e530e admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:42.900722 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9ffa85-d0bd-4959-a214-cc4a7f1e530e admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:42.901070 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9ffa85-d0bd-4959-a214-cc4a7f1e530e admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:42.901749 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9ffa85-d0bd-4959-a214-cc4a7f1e530e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:42.903369 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9ffa85-d0bd-4959-a214-cc4a7f1e530e admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:42.913622 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 782/1579] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:42 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:15:42.914749 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9ffa85-d0bd-4959-a214-cc4a7f1e530e admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:42.925036 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9ffa85-d0bd-4959-a214-cc4a7f1e530e admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:42.930837 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b15c65-68e7-4756-a717-c6eef579797f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:42.931680 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b15c65-68e7-4756-a717-c6eef579797f admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:42.931990 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b15c65-68e7-4756-a717-c6eef579797f admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:42.932266 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b15c65-68e7-4756-a717-c6eef579797f admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:42.932956 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b15c65-68e7-4756-a717-c6eef579797f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:42.934583 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b15c65-68e7-4756-a717-c6eef579797f admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:42.947121 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6267671b-abb1-40ed-8fef-538657c5cdf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:42.947384 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 783/1580] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:15:42.947729 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6267671b-abb1-40ed-8fef-538657c5cdf7 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:42.948004 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6267671b-abb1-40ed-8fef-538657c5cdf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:42.948124 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b15c65-68e7-4756-a717-c6eef579797f admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:42.948299 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6267671b-abb1-40ed-8fef-538657c5cdf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:42.948946 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6267671b-abb1-40ed-8fef-538657c5cdf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:42.950824 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6267671b-abb1-40ed-8fef-538657c5cdf7 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:42.962161 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 798/1581] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:42 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:15:42.962830 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6267671b-abb1-40ed-8fef-538657c5cdf7 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:42.965796 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b15c65-68e7-4756-a717-c6eef579797f admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:42.971132 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7fb147-3b54-4570-aed4-ef0660b217ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:42.971744 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7fb147-3b54-4570-aed4-ef0660b217ff admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:42.972023 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7fb147-3b54-4570-aed4-ef0660b217ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:42.972298 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7fb147-3b54-4570-aed4-ef0660b217ff admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:42.973097 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7fb147-3b54-4570-aed4-ef0660b217ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:42.974995 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7fb147-3b54-4570-aed4-ef0660b217ff admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:42.977721 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6267671b-abb1-40ed-8fef-538657c5cdf7 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:42.985132 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6d7196-fedc-4841-8428-5877cd7c707a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:42.985676 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6d7196-fedc-4841-8428-5877cd7c707a admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:42.986492 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6d7196-fedc-4841-8428-5877cd7c707a admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:42.986492 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6d7196-fedc-4841-8428-5877cd7c707a admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:42.987124 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6d7196-fedc-4841-8428-5877cd7c707a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:42.989675 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6d7196-fedc-4841-8428-5877cd7c707a admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:43.001106 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 784/1582] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:15:42 2022] POST /identity/v3/users => generated 287 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:15:43.002519 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841dac94-3358-4502-af4b-da194b46f951 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:43.003144 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841dac94-3358-4502-af4b-da194b46f951 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:43.003144 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841dac94-3358-4502-af4b-da194b46f951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:43.020233 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 799/1583] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:15:42 2022] POST /identity/v3/users => generated 288 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:15:43.021779 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6d7196-fedc-4841-8428-5877cd7c707a admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:43.023512 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-841dac94-3358-4502-af4b-da194b46f951 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:43.024704 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-841dac94-3358-4502-af4b-da194b46f951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:43.030213 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:43.033015 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 785/1584] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:15:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:43.033944 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841dac94-3358-4502-af4b-da194b46f951 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:43.035528 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6d7196-fedc-4841-8428-5877cd7c707a admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:43.044227 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841dac94-3358-4502-af4b-da194b46f951 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:43.045836 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae31824-436c-418c-9f34-879d7621c356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:43.046694 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae31824-436c-418c-9f34-879d7621c356 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:43.047133 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae31824-436c-418c-9f34-879d7621c356 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:43.047530 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae31824-436c-418c-9f34-879d7621c356 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:43.048389 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae31824-436c-418c-9f34-879d7621c356 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:43.049188 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3728ac-375b-4d3b-8ce6-2b9d7e57a849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:43.049886 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3728ac-375b-4d3b-8ce6-2b9d7e57a849 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:43.050167 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3728ac-375b-4d3b-8ce6-2b9d7e57a849 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:43.050435 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3728ac-375b-4d3b-8ce6-2b9d7e57a849 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:43.051067 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3728ac-375b-4d3b-8ce6-2b9d7e57a849 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:43.051609 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae31824-436c-418c-9f34-879d7621c356 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:43.053021 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3728ac-375b-4d3b-8ce6-2b9d7e57a849 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:43.078634 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 786/1585] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:15:43 2022] POST /identity/v3/users => generated 289 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:15:43.079300 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3728ac-375b-4d3b-8ce6-2b9d7e57a849 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:43.086409 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 800/1586] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:15:43 2022] POST /identity/v3/users => generated 287 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:15:43.087664 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae31824-436c-418c-9f34-879d7621c356 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:43.093762 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3728ac-375b-4d3b-8ce6-2b9d7e57a849 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:43.098583 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5a35c0-e82a-4760-8552-a8a57352fc16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:43.098832 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae31824-436c-418c-9f34-879d7621c356 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:43.099200 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5a35c0-e82a-4760-8552-a8a57352fc16 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:43.099479 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5a35c0-e82a-4760-8552-a8a57352fc16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:43.099747 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5a35c0-e82a-4760-8552-a8a57352fc16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:43.100429 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5a35c0-e82a-4760-8552-a8a57352fc16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:43.102114 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5a35c0-e82a-4760-8552-a8a57352fc16 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:43.105400 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4517a6d-583a-4204-a5eb-ba2f303b6df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:43.107018 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4517a6d-583a-4204-a5eb-ba2f303b6df2 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:43.107956 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4517a6d-583a-4204-a5eb-ba2f303b6df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:43.108575 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4517a6d-583a-4204-a5eb-ba2f303b6df2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:43.110077 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4517a6d-583a-4204-a5eb-ba2f303b6df2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:43.118560 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 787/1587] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:15:43.118819 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4517a6d-583a-4204-a5eb-ba2f303b6df2 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:43.118960 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5a35c0-e82a-4760-8552-a8a57352fc16 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:43.127303 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5a35c0-e82a-4760-8552-a8a57352fc16 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:43.132267 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585e55e1-73ea-4648-8bf1-6b8ea8f181cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:43.132955 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585e55e1-73ea-4648-8bf1-6b8ea8f181cb admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:43.133235 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585e55e1-73ea-4648-8bf1-6b8ea8f181cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:43.133595 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585e55e1-73ea-4648-8bf1-6b8ea8f181cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:43.139001 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 801/1588] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:15:43.139227 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585e55e1-73ea-4648-8bf1-6b8ea8f181cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:43.139326 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4517a6d-583a-4204-a5eb-ba2f303b6df2 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:43.139480 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585e55e1-73ea-4648-8bf1-6b8ea8f181cb admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:43.147402 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 788/1589] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:15:43.148278 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585e55e1-73ea-4648-8bf1-6b8ea8f181cb admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:43.153578 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:43.157585 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4517a6d-583a-4204-a5eb-ba2f303b6df2 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:43.162416 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585e55e1-73ea-4648-8bf1-6b8ea8f181cb admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:43.165621 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba55902-3268-49fb-8af8-f5ff1c81e014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:43.166559 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba55902-3268-49fb-8af8-f5ff1c81e014 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:43.166973 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba55902-3268-49fb-8af8-f5ff1c81e014 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:43.167401 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba55902-3268-49fb-8af8-f5ff1c81e014 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:43.168082 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba55902-3268-49fb-8af8-f5ff1c81e014 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:43.169746 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba55902-3268-49fb-8af8-f5ff1c81e014 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:43.171941 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c9129c-3b40-47a1-b6af-b635f6baccea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:43.173040 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c9129c-3b40-47a1-b6af-b635f6baccea admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:43.173480 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c9129c-3b40-47a1-b6af-b635f6baccea admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:43.173918 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c9129c-3b40-47a1-b6af-b635f6baccea admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:43.175027 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c9129c-3b40-47a1-b6af-b635f6baccea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:43.177411 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c9129c-3b40-47a1-b6af-b635f6baccea admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:43.182780 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 802/1590] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:15:43.183844 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba55902-3268-49fb-8af8-f5ff1c81e014 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:43.195109 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 789/1591] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:15:43.196366 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c9129c-3b40-47a1-b6af-b635f6baccea admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:43.211681 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c9129c-3b40-47a1-b6af-b635f6baccea admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:43.220721 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d0094b-8838-463c-8261-28617738ea6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:43.221809 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d0094b-8838-463c-8261-28617738ea6c admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:43.222321 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d0094b-8838-463c-8261-28617738ea6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:43.222838 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d0094b-8838-463c-8261-28617738ea6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:43.223931 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d0094b-8838-463c-8261-28617738ea6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:43.226754 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d0094b-8838-463c-8261-28617738ea6c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:43.243691 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 790/1592] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:15:43.244055 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d0094b-8838-463c-8261-28617738ea6c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:43.253766 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba55902-3268-49fb-8af8-f5ff1c81e014 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:43.257815 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d90d5ba-dffc-46c0-8487-5c098035ddae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:43.258521 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d90d5ba-dffc-46c0-8487-5c098035ddae service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:43.258771 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d90d5ba-dffc-46c0-8487-5c098035ddae service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:43.259047 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d90d5ba-dffc-46c0-8487-5c098035ddae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:43.259723 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d0094b-8838-463c-8261-28617738ea6c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:43.264601 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1a7dfe-d93d-4ace-9504-00b6786e85d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:43.265267 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a7dfe-d93d-4ace-9504-00b6786e85d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:43.265548 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a7dfe-d93d-4ace-9504-00b6786e85d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:43.265817 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a7dfe-d93d-4ace-9504-00b6786e85d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:43.266468 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1a7dfe-d93d-4ace-9504-00b6786e85d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:43.268235 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1a7dfe-d93d-4ace-9504-00b6786e85d4 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:43.269819 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d90d5ba-dffc-46c0-8487-5c098035ddae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:43.271507 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d90d5ba-dffc-46c0-8487-5c098035ddae service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:43.283102 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 803/1593] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:15:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:15:43.283650 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d90d5ba-dffc-46c0-8487-5c098035ddae service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:43.287380 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 791/1594] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:15:43.288670 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1a7dfe-d93d-4ace-9504-00b6786e85d4 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:43.296027 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d90d5ba-dffc-46c0-8487-5c098035ddae service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:43.301049 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1a7dfe-d93d-4ace-9504-00b6786e85d4 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:43.302570 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6930a9-5234-4fc9-8214-9c21db7af7b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:43.303455 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6930a9-5234-4fc9-8214-9c21db7af7b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:43.303761 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6930a9-5234-4fc9-8214-9c21db7af7b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:43.304216 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6930a9-5234-4fc9-8214-9c21db7af7b7 admin admin] PATH_INFO: `/v3/projects/8b93a6978c404ca4aa497919103b5283/users/c3a65f26ab214feab60c7b6380539076/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:43.308113 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a0800e-6b05-4eb7-a1f5-3586eebac09a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:43.309305 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0800e-6b05-4eb7-a1f5-3586eebac09a admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:43.309776 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0800e-6b05-4eb7-a1f5-3586eebac09a admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:43.310226 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0800e-6b05-4eb7-a1f5-3586eebac09a admin admin] PATH_INFO: `/v3/projects/71de7da6100247d6a8defff682678e1f/users/9ecc6495a297447b9e0239a1c91a2f3d/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:43.323384 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6930a9-5234-4fc9-8214-9c21db7af7b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b93a6978c404ca4aa497919103b5283, user_id=c3a65f26ab214feab60c7b6380539076, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:43.326220 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6930a9-5234-4fc9-8214-9c21db7af7b7 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:43.328737 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a0800e-6b05-4eb7-a1f5-3586eebac09a admin admin] RBAC: Authorizing `identity:create_grant(project_id=71de7da6100247d6a8defff682678e1f, user_id=9ecc6495a297447b9e0239a1c91a2f3d, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:43.329715 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: WARNING py.warnings [None req-a1a0800e-6b05-4eb7-a1f5-3586eebac09a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 16:15:43.329715 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: warnings.warn(msg) Sep 06 16:15:43.329715 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: Sep 06 16:15:43.331765 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a0800e-6b05-4eb7-a1f5-3586eebac09a admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:43.343729 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 804/1595] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:15:43 2022] PUT /identity/v3/projects/8b93a6978c404ca4aa497919103b5283/users/c3a65f26ab214feab60c7b6380539076/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:43.349513 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6930a9-5234-4fc9-8214-9c21db7af7b7 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:43.352420 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 792/1596] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:15:43 2022] PUT /identity/v3/projects/71de7da6100247d6a8defff682678e1f/users/9ecc6495a297447b9e0239a1c91a2f3d/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:43.353577 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a0800e-6b05-4eb7-a1f5-3586eebac09a admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:43.442882 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a0800e-6b05-4eb7-a1f5-3586eebac09a admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:43.449405 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d7a96c-c052-438f-b575-5e74cc9d60d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:43.450489 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d7a96c-c052-438f-b575-5e74cc9d60d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:43.450888 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d7a96c-c052-438f-b575-5e74cc9d60d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:43.451227 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d7a96c-c052-438f-b575-5e74cc9d60d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:43.452135 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d7a96c-c052-438f-b575-5e74cc9d60d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:43.455010 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d7a96c-c052-438f-b575-5e74cc9d60d8 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:43.458477 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6930a9-5234-4fc9-8214-9c21db7af7b7 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:43.465344 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af356a3-09a9-482e-806d-2143e3d499a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:43.466632 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af356a3-09a9-482e-806d-2143e3d499a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:43.466632 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af356a3-09a9-482e-806d-2143e3d499a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:43.466632 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af356a3-09a9-482e-806d-2143e3d499a7 admin admin] PATH_INFO: `/v3/projects/49a3e0cfba224a119bf8aaf26ae583f4/users/93096a58fcb34f2085a05ffa40df5937/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:43.467377 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 793/1597] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:15:43.468320 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d7a96c-c052-438f-b575-5e74cc9d60d8 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:43.482053 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d7a96c-c052-438f-b575-5e74cc9d60d8 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:43.489564 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ba5b76-a34a-4663-bd24-509894bebe16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:43.490378 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ba5b76-a34a-4663-bd24-509894bebe16 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:43.490801 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ba5b76-a34a-4663-bd24-509894bebe16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:43.491408 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ba5b76-a34a-4663-bd24-509894bebe16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:43.491985 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af356a3-09a9-482e-806d-2143e3d499a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49a3e0cfba224a119bf8aaf26ae583f4, user_id=93096a58fcb34f2085a05ffa40df5937, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:43.492563 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ba5b76-a34a-4663-bd24-509894bebe16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:43.494763 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af356a3-09a9-482e-806d-2143e3d499a7 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:43.495437 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ba5b76-a34a-4663-bd24-509894bebe16 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:43.516903 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 794/1598] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:15:43.518256 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ba5b76-a34a-4663-bd24-509894bebe16 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:43.524373 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 805/1599] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:15:43 2022] PUT /identity/v3/projects/49a3e0cfba224a119bf8aaf26ae583f4/users/93096a58fcb34f2085a05ffa40df5937/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:43.525959 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af356a3-09a9-482e-806d-2143e3d499a7 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:43.533457 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ba5b76-a34a-4663-bd24-509894bebe16 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:43.565984 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:43.609518 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af356a3-09a9-482e-806d-2143e3d499a7 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:43.615086 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2824c2-5fa2-44fa-b2fc-4d755b47d453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:43.616089 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2824c2-5fa2-44fa-b2fc-4d755b47d453 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:43.616569 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2824c2-5fa2-44fa-b2fc-4d755b47d453 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:43.616969 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2824c2-5fa2-44fa-b2fc-4d755b47d453 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:43.618029 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2824c2-5fa2-44fa-b2fc-4d755b47d453 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:43.619865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2824c2-5fa2-44fa-b2fc-4d755b47d453 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:43.625267 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b770b2f5-2e38-4ed1-80a5-36f80eb21fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:43.627175 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b770b2f5-2e38-4ed1-80a5-36f80eb21fca admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:43.627893 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b770b2f5-2e38-4ed1-80a5-36f80eb21fca admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:43.628715 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b770b2f5-2e38-4ed1-80a5-36f80eb21fca admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:43.629538 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b770b2f5-2e38-4ed1-80a5-36f80eb21fca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:43.631199 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b770b2f5-2e38-4ed1-80a5-36f80eb21fca admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:43.633364 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 806/1600] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:15:43.634687 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dda3787-c5ff-4080-92e7-59ee9166a457 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:43.635020 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dda3787-c5ff-4080-92e7-59ee9166a457 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:43.635335 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dda3787-c5ff-4080-92e7-59ee9166a457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:43.646298 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 795/1601] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:15:43.646922 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20a6882-0598-464a-93d6-cfe895be1dec None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:43.647203 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20a6882-0598-464a-93d6-cfe895be1dec None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:43.647470 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20a6882-0598-464a-93d6-cfe895be1dec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:43.662593 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-9dda3787-c5ff-4080-92e7-59ee9166a457 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:43.663834 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-9dda3787-c5ff-4080-92e7-59ee9166a457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:43.668085 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-d20a6882-0598-464a-93d6-cfe895be1dec None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:43.669350 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d20a6882-0598-464a-93d6-cfe895be1dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:43.675961 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 807/1602] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:15:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:43.677365 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dda3787-c5ff-4080-92e7-59ee9166a457 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:43.691186 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 796/1603] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:15:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:43.692238 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20a6882-0598-464a-93d6-cfe895be1dec None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:43.704860 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dda3787-c5ff-4080-92e7-59ee9166a457 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:43.707544 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20a6882-0598-464a-93d6-cfe895be1dec None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:43.713199 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9555727-f39f-49f4-89ba-68e7d779a4df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:43.713776 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d735d1-18ca-447f-99da-aa16d6f75f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:43.714367 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9555727-f39f-49f4-89ba-68e7d779a4df admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:43.714537 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d735d1-18ca-447f-99da-aa16d6f75f33 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:43.714634 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9555727-f39f-49f4-89ba-68e7d779a4df admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:43.714862 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d735d1-18ca-447f-99da-aa16d6f75f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:43.715037 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9555727-f39f-49f4-89ba-68e7d779a4df admin admin] PATH_INFO: `/v3/projects/b1eddfa78b5745cb86be669ffeea8c97/users/dc0dc02045fd49a69e4accdd626e01a7/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:43.715223 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d735d1-18ca-447f-99da-aa16d6f75f33 admin admin] PATH_INFO: `/v3/projects/19286596a2414ae78ba7080b971364ca/users/f180909caf24492db646b89ebd8ca214/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:43.735193 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d735d1-18ca-447f-99da-aa16d6f75f33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19286596a2414ae78ba7080b971364ca, user_id=f180909caf24492db646b89ebd8ca214, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:43.735748 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9555727-f39f-49f4-89ba-68e7d779a4df admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1eddfa78b5745cb86be669ffeea8c97, user_id=dc0dc02045fd49a69e4accdd626e01a7, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:43.738329 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d735d1-18ca-447f-99da-aa16d6f75f33 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:43.739052 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9555727-f39f-49f4-89ba-68e7d779a4df admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:43.745967 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:43.763282 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 808/1604] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:15:43 2022] PUT /identity/v3/projects/19286596a2414ae78ba7080b971364ca/users/f180909caf24492db646b89ebd8ca214/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:43.765206 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e497861-e88a-4e93-a7e4-fe46a4b5dd60 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:43.765354 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e497861-e88a-4e93-a7e4-fe46a4b5dd60 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:43.765722 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e497861-e88a-4e93-a7e4-fe46a4b5dd60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:43.767790 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 797/1605] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:15:43 2022] PUT /identity/v3/projects/b1eddfa78b5745cb86be669ffeea8c97/users/dc0dc02045fd49a69e4accdd626e01a7/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:43.769353 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9555727-f39f-49f4-89ba-68e7d779a4df admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:43.811352 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-5e497861-e88a-4e93-a7e4-fe46a4b5dd60 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:43.817242 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5e497861-e88a-4e93-a7e4-fe46a4b5dd60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:43.879439 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9555727-f39f-49f4-89ba-68e7d779a4df admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:43.886735 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4b67e1-674d-4118-945f-04b9960ea2d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:43.887790 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4b67e1-674d-4118-945f-04b9960ea2d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:43.888214 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4b67e1-674d-4118-945f-04b9960ea2d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:43.888629 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4b67e1-674d-4118-945f-04b9960ea2d4 admin admin] PATH_INFO: `/v3/projects/29659e2d27344b079fa71e73e707726d/users/9171d03379714b13a4b3fdd64a54f56a/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:43.913624 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4b67e1-674d-4118-945f-04b9960ea2d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29659e2d27344b079fa71e73e707726d, user_id=9171d03379714b13a4b3fdd64a54f56a, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:43.921872 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4b67e1-674d-4118-945f-04b9960ea2d4 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:43.923762 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 809/1606] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:15:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:43.925363 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e497861-e88a-4e93-a7e4-fe46a4b5dd60 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:43.942657 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e497861-e88a-4e93-a7e4-fe46a4b5dd60 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:43.948608 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 798/1607] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:15:43 2022] PUT /identity/v3/projects/29659e2d27344b079fa71e73e707726d/users/9171d03379714b13a4b3fdd64a54f56a/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:43.950400 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4b67e1-674d-4118-945f-04b9960ea2d4 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:44.041429 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4b67e1-674d-4118-945f-04b9960ea2d4 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:44.047048 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b678216a-f474-405a-b4ad-9fedada82ba3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:44.047691 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b678216a-f474-405a-b4ad-9fedada82ba3 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:44.048049 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b678216a-f474-405a-b4ad-9fedada82ba3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:44.048251 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b678216a-f474-405a-b4ad-9fedada82ba3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:44.048867 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5e003e-b26f-43ba-af45-18c6532f45ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:44.050320 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5e003e-b26f-43ba-af45-18c6532f45ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:44.050961 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5e003e-b26f-43ba-af45-18c6532f45ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:44.051384 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5e003e-b26f-43ba-af45-18c6532f45ab admin admin] PATH_INFO: `/v3/projects/c66250df2e054415b517b2de8857bc87/users/8978e7cc78ab4c559dfd3e22cde34569/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:44.064299 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b678216a-f474-405a-b4ad-9fedada82ba3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:44.066806 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b678216a-f474-405a-b4ad-9fedada82ba3 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:44.090844 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5e003e-b26f-43ba-af45-18c6532f45ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=c66250df2e054415b517b2de8857bc87, user_id=8978e7cc78ab4c559dfd3e22cde34569, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:44.096043 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5e003e-b26f-43ba-af45-18c6532f45ab admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:44.100837 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 799/1608] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:15:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:15:44.103570 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b678216a-f474-405a-b4ad-9fedada82ba3 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:44.119688 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b678216a-f474-405a-b4ad-9fedada82ba3 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:44.138243 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 810/1609] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:15:43 2022] PUT /identity/v3/projects/c66250df2e054415b517b2de8857bc87/users/8978e7cc78ab4c559dfd3e22cde34569/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:44.148937 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf43e6e-87fa-4d88-acb6-94ba80b2fa7f None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:44.149434 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf43e6e-87fa-4d88-acb6-94ba80b2fa7f None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:44.149752 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf43e6e-87fa-4d88-acb6-94ba80b2fa7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:44.205065 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50c44bd-2aca-4dc7-a631-b3eeb338287d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:44.205889 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50c44bd-2aca-4dc7-a631-b3eeb338287d service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:44.206337 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50c44bd-2aca-4dc7-a631-b3eeb338287d service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:44.206699 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50c44bd-2aca-4dc7-a631-b3eeb338287d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:44.217527 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-cbf43e6e-87fa-4d88-acb6-94ba80b2fa7f None None] MFA Rules not processed for user `f180909caf24492db646b89ebd8ca214`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:44.219256 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-cbf43e6e-87fa-4d88-acb6-94ba80b2fa7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:44.226215 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:44.300369 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:44.385264 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50c44bd-2aca-4dc7-a631-b3eeb338287d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:44.386751 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50c44bd-2aca-4dc7-a631-b3eeb338287d service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:44.387402 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 811/1610] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:15:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:44.401240 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30673f09-1a1b-462d-927a-e474a6d5f8c7 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:44.403254 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30673f09-1a1b-462d-927a-e474a6d5f8c7 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:44.403254 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30673f09-1a1b-462d-927a-e474a6d5f8c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:44.429951 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 800/1611] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:15:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:15:44.431285 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50c44bd-2aca-4dc7-a631-b3eeb338287d service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:44.453286 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50c44bd-2aca-4dc7-a631-b3eeb338287d service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:44.463041 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ea61df-6789-4a6a-a926-63c304544fa7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:44.463885 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ea61df-6789-4a6a-a926-63c304544fa7 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:44.464520 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ea61df-6789-4a6a-a926-63c304544fa7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:44.467372 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ea61df-6789-4a6a-a926-63c304544fa7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:44.478888 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ea61df-6789-4a6a-a926-63c304544fa7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:44.482361 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ea61df-6789-4a6a-a926-63c304544fa7 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:44.494089 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-30673f09-1a1b-462d-927a-e474a6d5f8c7 None None] MFA Rules not processed for user `dc0dc02045fd49a69e4accdd626e01a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:44.495947 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-30673f09-1a1b-462d-927a-e474a6d5f8c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:44.507230 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 801/1612] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:15:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:15:44.508804 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7c5a6-c0a3-4200-bdbb-faa09dddab6a None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:44.511993 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7c5a6-c0a3-4200-bdbb-faa09dddab6a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:44.512498 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7c5a6-c0a3-4200-bdbb-faa09dddab6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:44.560458 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-9cf7c5a6-c0a3-4200-bdbb-faa09dddab6a None None] MFA Rules not processed for user `9171d03379714b13a4b3fdd64a54f56a`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:44.561819 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-9cf7c5a6-c0a3-4200-bdbb-faa09dddab6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:44.654005 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 812/1613] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:15:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:44.656249 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28c6ad9-e545-423f-9ece-0ee8bc24b272 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:44.656786 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28c6ad9-e545-423f-9ece-0ee8bc24b272 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:44.657169 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28c6ad9-e545-423f-9ece-0ee8bc24b272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:44.684834 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 802/1614] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:15:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:44.686281 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab1fe55-d653-4d01-b6c9-1cfe7829f4ea None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:44.686571 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab1fe55-d653-4d01-b6c9-1cfe7829f4ea None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:44.686842 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab1fe55-d653-4d01-b6c9-1cfe7829f4ea None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:44.691935 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 803/1615] 10.222.0.43 () {54 vars in 962 bytes} [Tue Sep 6 16:15:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:15:44.692851 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab1fe55-d653-4d01-b6c9-1cfe7829f4ea None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:44.703895 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-f28c6ad9-e545-423f-9ece-0ee8bc24b272 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:44.705773 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f28c6ad9-e545-423f-9ece-0ee8bc24b272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:44.727181 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 813/1616] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:15:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:44.728097 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab1fe55-d653-4d01-b6c9-1cfe7829f4ea None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:44.728918 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28c6ad9-e545-423f-9ece-0ee8bc24b272 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:44.740829 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6c34d7-2895-4fc6-aa42-e76adf820250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:44.741450 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6c34d7-2895-4fc6-aa42-e76adf820250 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:44.741809 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6c34d7-2895-4fc6-aa42-e76adf820250 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:44.742074 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6c34d7-2895-4fc6-aa42-e76adf820250 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:44.742826 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6c34d7-2895-4fc6-aa42-e76adf820250 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:44.744903 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6c34d7-2895-4fc6-aa42-e76adf820250 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:44.746600 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28c6ad9-e545-423f-9ece-0ee8bc24b272 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:44.752455 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d9ef44-b126-4050-89d4-34666151a2b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:44.753905 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d9ef44-b126-4050-89d4-34666151a2b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:44.754283 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d9ef44-b126-4050-89d4-34666151a2b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:44.754685 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d9ef44-b126-4050-89d4-34666151a2b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:44.756451 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d9ef44-b126-4050-89d4-34666151a2b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:44.761008 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d9ef44-b126-4050-89d4-34666151a2b6 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:44.789031 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 804/1617] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:15:44 2022] POST /identity/v3/projects => generated 377 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:15:44.794102 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc5887d-b401-43b9-ab8e-ed2710ede0f6 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:44.794697 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc5887d-b401-43b9-ab8e-ed2710ede0f6 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:44.795450 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc5887d-b401-43b9-ab8e-ed2710ede0f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:44.825136 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 814/1618] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:15:44 2022] POST /identity/v3/projects => generated 371 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:15:44.827032 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d9ef44-b126-4050-89d4-34666151a2b6 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:44.862510 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-ffc5887d-b401-43b9-ab8e-ed2710ede0f6 None None] MFA Rules not processed for user `63de41f4348243998029f7d07aa5dc40`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:44.863916 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ffc5887d-b401-43b9-ab8e-ed2710ede0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:44.931477 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d9ef44-b126-4050-89d4-34666151a2b6 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:44.938519 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c352155b-f14f-44a7-b994-b8793e624d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:44.939657 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c352155b-f14f-44a7-b994-b8793e624d0d admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:44.940109 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c352155b-f14f-44a7-b994-b8793e624d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:44.940521 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c352155b-f14f-44a7-b994-b8793e624d0d admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:44.941719 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c352155b-f14f-44a7-b994-b8793e624d0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:44.944250 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c352155b-f14f-44a7-b994-b8793e624d0d admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:44.958530 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:44.965803 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 805/1619] 10.222.0.43 () {58 vars in 1059 bytes} [Tue Sep 6 16:15:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:44.966939 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc5887d-b401-43b9-ab8e-ed2710ede0f6 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:44.974269 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 815/1620] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:15:44 2022] POST /identity/v3/users => generated 295 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:15:44.975459 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c352155b-f14f-44a7-b994-b8793e624d0d admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:44.978246 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:44.986889 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc5887d-b401-43b9-ab8e-ed2710ede0f6 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:44.992796 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090304f2-8727-4961-a7db-4d9e0a15f320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:44.994271 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090304f2-8727-4961-a7db-4d9e0a15f320 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:44.994732 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090304f2-8727-4961-a7db-4d9e0a15f320 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:44.995063 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090304f2-8727-4961-a7db-4d9e0a15f320 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:44.995775 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090304f2-8727-4961-a7db-4d9e0a15f320 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:44.999128 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090304f2-8727-4961-a7db-4d9e0a15f320 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:45.033440 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 806/1621] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:15:44 2022] POST /identity/v3/users => generated 292 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:15:45.034219 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090304f2-8727-4961-a7db-4d9e0a15f320 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:45.052930 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090304f2-8727-4961-a7db-4d9e0a15f320 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:45.060794 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc5cc57-1491-4df5-9e93-23ba0e8748cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63de41f4348243998029f7d07aa5dc40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:45.061598 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc5cc57-1491-4df5-9e93-23ba0e8748cd service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:45.062016 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc5cc57-1491-4df5-9e93-23ba0e8748cd service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:45.062346 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc5cc57-1491-4df5-9e93-23ba0e8748cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:45.064889 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c352155b-f14f-44a7-b994-b8793e624d0d admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:45.069070 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526e5189-1c49-4197-9307-478a983772ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:45.070047 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526e5189-1c49-4197-9307-478a983772ce service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:45.070398 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526e5189-1c49-4197-9307-478a983772ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:45.070799 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526e5189-1c49-4197-9307-478a983772ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:45.083061 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526e5189-1c49-4197-9307-478a983772ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:45.084194 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526e5189-1c49-4197-9307-478a983772ce service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:45.097180 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 816/1622] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:15:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:15:45.098635 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526e5189-1c49-4197-9307-478a983772ce service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:45.112715 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526e5189-1c49-4197-9307-478a983772ce service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:45.123134 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ca4e0c-bf7e-4235-ae5a-d194c28f64e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:45.123767 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ca4e0c-bf7e-4235-ae5a-d194c28f64e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:45.124122 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ca4e0c-bf7e-4235-ae5a-d194c28f64e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:45.125093 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ca4e0c-bf7e-4235-ae5a-d194c28f64e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:45.126038 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ca4e0c-bf7e-4235-ae5a-d194c28f64e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:45.127908 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ca4e0c-bf7e-4235-ae5a-d194c28f64e5 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:45.150730 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 817/1623] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:45 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:15:45.151904 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ca4e0c-bf7e-4235-ae5a-d194c28f64e5 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:45.174922 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc5cc57-1491-4df5-9e93-23ba0e8748cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:45.176614 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc5cc57-1491-4df5-9e93-23ba0e8748cd service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:45.177135 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ca4e0c-bf7e-4235-ae5a-d194c28f64e5 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:45.187832 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88cc064-7e09-4b90-b731-dca4fdd352e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:45.188456 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88cc064-7e09-4b90-b731-dca4fdd352e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:45.189075 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88cc064-7e09-4b90-b731-dca4fdd352e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:45.190507 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88cc064-7e09-4b90-b731-dca4fdd352e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:45.191179 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88cc064-7e09-4b90-b731-dca4fdd352e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:45.198164 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88cc064-7e09-4b90-b731-dca4fdd352e3 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:45.199512 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 807/1624] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:15:45 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:15:45.214762 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc5cc57-1491-4df5-9e93-23ba0e8748cd service nova] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:45.253068 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 818/1625] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:45 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:15:45.254040 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88cc064-7e09-4b90-b731-dca4fdd352e3 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:45.256794 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc5cc57-1491-4df5-9e93-23ba0e8748cd service nova] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:45.282954 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f95f2d-c0da-4aea-9ded-7180ffc390f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:45.283850 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88cc064-7e09-4b90-b731-dca4fdd352e3 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:45.284442 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f95f2d-c0da-4aea-9ded-7180ffc390f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:45.285181 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f95f2d-c0da-4aea-9ded-7180ffc390f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:45.285789 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f95f2d-c0da-4aea-9ded-7180ffc390f9 admin admin] PATH_INFO: `/v3/projects/2dfa02e2744a4e098f57ba1b9e75226a/users/858442ac71bb43998f820c8476024e46/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:45.297518 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73770d60-407b-4a21-8172-39d573e7737a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:45.298212 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73770d60-407b-4a21-8172-39d573e7737a admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:45.299414 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73770d60-407b-4a21-8172-39d573e7737a admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:45.299745 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73770d60-407b-4a21-8172-39d573e7737a admin admin] PATH_INFO: `/v3/projects/8cce38d3f06f45dba538a8940c63efc6/users/34d1a67a7d3a4802abca489173c4ca95/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:45.335311 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f95f2d-c0da-4aea-9ded-7180ffc390f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2dfa02e2744a4e098f57ba1b9e75226a, user_id=858442ac71bb43998f820c8476024e46, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:45.335802 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73770d60-407b-4a21-8172-39d573e7737a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cce38d3f06f45dba538a8940c63efc6, user_id=34d1a67a7d3a4802abca489173c4ca95, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:45.339002 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73770d60-407b-4a21-8172-39d573e7737a admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:45.342842 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f95f2d-c0da-4aea-9ded-7180ffc390f9 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:45.366531 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 819/1626] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:15:45 2022] PUT /identity/v3/projects/8cce38d3f06f45dba538a8940c63efc6/users/34d1a67a7d3a4802abca489173c4ca95/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:45.371582 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73770d60-407b-4a21-8172-39d573e7737a admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:45.380439 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 808/1627] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:15:45 2022] PUT /identity/v3/projects/2dfa02e2744a4e098f57ba1b9e75226a/users/858442ac71bb43998f820c8476024e46/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:45.384875 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f95f2d-c0da-4aea-9ded-7180ffc390f9 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:45.536074 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f95f2d-c0da-4aea-9ded-7180ffc390f9 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:45.548164 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd53ef6-b010-4b9d-b61d-acc6b8403130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:45.548838 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd53ef6-b010-4b9d-b61d-acc6b8403130 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:45.549121 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd53ef6-b010-4b9d-b61d-acc6b8403130 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:45.549462 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd53ef6-b010-4b9d-b61d-acc6b8403130 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:45.550119 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd53ef6-b010-4b9d-b61d-acc6b8403130 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:45.551930 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd53ef6-b010-4b9d-b61d-acc6b8403130 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:45.564022 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 809/1628] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:45 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:15:45.568392 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd53ef6-b010-4b9d-b61d-acc6b8403130 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:45.587306 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73770d60-407b-4a21-8172-39d573e7737a admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:45.591128 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd53ef6-b010-4b9d-b61d-acc6b8403130 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:45.600711 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eee1383-abf4-4da0-93a0-859a867cfb79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:45.601375 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee1383-abf4-4da0-93a0-859a867cfb79 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:45.601650 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee1383-abf4-4da0-93a0-859a867cfb79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:45.601926 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee1383-abf4-4da0-93a0-859a867cfb79 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:45.603622 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eee1383-abf4-4da0-93a0-859a867cfb79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:45.605236 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ca1ffd-00c3-4ca0-ae05-e3d4d603a9fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:45.606074 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eee1383-abf4-4da0-93a0-859a867cfb79 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:45.606857 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ca1ffd-00c3-4ca0-ae05-e3d4d603a9fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:45.607401 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ca1ffd-00c3-4ca0-ae05-e3d4d603a9fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:45.608286 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ca1ffd-00c3-4ca0-ae05-e3d4d603a9fc admin admin] PATH_INFO: `/v3/projects/2dfa02e2744a4e098f57ba1b9e75226a/users/858442ac71bb43998f820c8476024e46/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:45.611216 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ca1ffd-00c3-4ca0-ae05-e3d4d603a9fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=2dfa02e2744a4e098f57ba1b9e75226a, user_id=858442ac71bb43998f820c8476024e46, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:45.619341 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ca1ffd-00c3-4ca0-ae05-e3d4d603a9fc admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:45.625355 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 820/1629] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:45 2022] GET /identity/v3/roles => generated 1420 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:15:45.627187 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eee1383-abf4-4da0-93a0-859a867cfb79 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:45.642134 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eee1383-abf4-4da0-93a0-859a867cfb79 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:45.642523 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 810/1630] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:15:45 2022] PUT /identity/v3/projects/2dfa02e2744a4e098f57ba1b9e75226a/users/858442ac71bb43998f820c8476024e46/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:45.653358 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf9d76e-ae5c-4ca8-bdbb-44bfc444435e None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:45.654993 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf9d76e-ae5c-4ca8-bdbb-44bfc444435e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:45.655793 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf9d76e-ae5c-4ca8-bdbb-44bfc444435e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:45.728717 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-daf9d76e-ae5c-4ca8-bdbb-44bfc444435e None None] MFA Rules not processed for user `858442ac71bb43998f820c8476024e46`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:45.732955 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-daf9d76e-ae5c-4ca8-bdbb-44bfc444435e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:45.770923 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:45.794587 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34de2d3d-f76a-4f79-a996-9751d1bcada2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:45.795453 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34de2d3d-f76a-4f79-a996-9751d1bcada2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:45.795947 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34de2d3d-f76a-4f79-a996-9751d1bcada2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:45.796387 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34de2d3d-f76a-4f79-a996-9751d1bcada2 admin admin] PATH_INFO: `/v3/projects/8cce38d3f06f45dba538a8940c63efc6/users/34d1a67a7d3a4802abca489173c4ca95/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:45.799586 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34de2d3d-f76a-4f79-a996-9751d1bcada2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cce38d3f06f45dba538a8940c63efc6, user_id=34d1a67a7d3a4802abca489173c4ca95, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:45.803921 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34de2d3d-f76a-4f79-a996-9751d1bcada2 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:45.839055 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 821/1631] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:15:45 2022] PUT /identity/v3/projects/8cce38d3f06f45dba538a8940c63efc6/users/34d1a67a7d3a4802abca489173c4ca95/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:45.843793 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:45.854814 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5c5fbe-b929-47da-b880-b81e723b1b47 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:45.855457 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5c5fbe-b929-47da-b880-b81e723b1b47 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:45.856015 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5c5fbe-b929-47da-b880-b81e723b1b47 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:45.860573 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 822/1632] 10.222.0.43 () {54 vars in 912 bytes} [Tue Sep 6 16:15:45 2022] GET /identity => generated 270 bytes in 20 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:15:45.862696 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c33d6af-e3e3-4462-9765-3efc9792b2ed None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:45.865188 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c33d6af-e3e3-4462-9765-3efc9792b2ed None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:45.865188 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c33d6af-e3e3-4462-9765-3efc9792b2ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:45.967951 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-6c33d6af-e3e3-4462-9765-3efc9792b2ed None None] MFA Rules not processed for user `34d1a67a7d3a4802abca489173c4ca95`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:45.970734 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-6c33d6af-e3e3-4462-9765-3efc9792b2ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:45.972461 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 811/1633] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:15:45 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:15:45.974211 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a66aa-f397-4ac5-8ea7-c38b4d118b7a None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:45.974544 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a66aa-f397-4ac5-8ea7-c38b4d118b7a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:45.975455 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a66aa-f397-4ac5-8ea7-c38b4d118b7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:46.041313 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-2e3a66aa-f397-4ac5-8ea7-c38b4d118b7a None None] MFA Rules not processed for user `63de41f4348243998029f7d07aa5dc40`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:46.045282 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-2e3a66aa-f397-4ac5-8ea7-c38b4d118b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:46.177985 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 823/1634] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:15:45 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:46.177985 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c33d6af-e3e3-4462-9765-3efc9792b2ed None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:46.230272 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 812/1635] 10.222.0.43 () {58 vars in 1009 bytes} [Tue Sep 6 16:15:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:46.231184 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3a66aa-f397-4ac5-8ea7-c38b4d118b7a None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:46.328381 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c33d6af-e3e3-4462-9765-3efc9792b2ed None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:46.333255 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbea89b-a8cf-4344-9bcf-333ddfd5095b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:46.333931 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbea89b-a8cf-4344-9bcf-333ddfd5095b service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:46.334219 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbea89b-a8cf-4344-9bcf-333ddfd5095b service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:46.334700 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbea89b-a8cf-4344-9bcf-333ddfd5095b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:46.390185 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3a66aa-f397-4ac5-8ea7-c38b4d118b7a None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:46.400192 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eb8438-67f7-4768-b745-23344b90cf1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:46.401047 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eb8438-67f7-4768-b745-23344b90cf1c service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:46.401418 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eb8438-67f7-4768-b745-23344b90cf1c service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:46.401780 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eb8438-67f7-4768-b745-23344b90cf1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:46.424269 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eb8438-67f7-4768-b745-23344b90cf1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:46.426678 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eb8438-67f7-4768-b745-23344b90cf1c service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:46.431210 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbea89b-a8cf-4344-9bcf-333ddfd5095b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:46.432703 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbea89b-a8cf-4344-9bcf-333ddfd5095b service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:46.445318 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 824/1636] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:15:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:15:46.454888 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 813/1637] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:15:46 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:15:46.566103 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbea89b-a8cf-4344-9bcf-333ddfd5095b service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:46.576098 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbea89b-a8cf-4344-9bcf-333ddfd5095b service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:46.586260 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184afa2e-d148-4230-9d44-91617b2af311 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:46.587080 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184afa2e-d148-4230-9d44-91617b2af311 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:46.587459 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184afa2e-d148-4230-9d44-91617b2af311 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:46.587825 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184afa2e-d148-4230-9d44-91617b2af311 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:46.601738 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184afa2e-d148-4230-9d44-91617b2af311 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:46.603860 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184afa2e-d148-4230-9d44-91617b2af311 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:46.622313 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 825/1638] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:15:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:15:47.123779 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c879174e-36cd-4f62-9441-61b55ad84566 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:47.124790 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c879174e-36cd-4f62-9441-61b55ad84566 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:47.124790 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c879174e-36cd-4f62-9441-61b55ad84566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:47.223358 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-c879174e-36cd-4f62-9441-61b55ad84566 None None] MFA Rules not processed for user `9ecc6495a297447b9e0239a1c91a2f3d`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:47.225434 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-c879174e-36cd-4f62-9441-61b55ad84566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:47.443360 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:47.490304 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 814/1639] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:15:47 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 367 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:47.496925 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184afa2e-d148-4230-9d44-91617b2af311 service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:47.523651 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:47.723721 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184afa2e-d148-4230-9d44-91617b2af311 service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:47.743834 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547a1787-18a3-4ad0-8570-83d428bf3976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:47.744792 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547a1787-18a3-4ad0-8570-83d428bf3976 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:47.745287 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547a1787-18a3-4ad0-8570-83d428bf3976 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:47.745763 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547a1787-18a3-4ad0-8570-83d428bf3976 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:47.746906 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547a1787-18a3-4ad0-8570-83d428bf3976 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:47.755808 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547a1787-18a3-4ad0-8570-83d428bf3976 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:47.866001 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 826/1640] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:15:47 2022] POST /identity/v3/projects => generated 403 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:15:47.872619 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c879174e-36cd-4f62-9441-61b55ad84566 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:47.911603 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c879174e-36cd-4f62-9441-61b55ad84566 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:48.111634 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547a1787-18a3-4ad0-8570-83d428bf3976 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:48.131916 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff2dbaa-73fa-491b-b047-dbe34883e403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:48.132938 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff2dbaa-73fa-491b-b047-dbe34883e403 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:48.133537 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff2dbaa-73fa-491b-b047-dbe34883e403 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:48.133910 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff2dbaa-73fa-491b-b047-dbe34883e403 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:48.134814 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff2dbaa-73fa-491b-b047-dbe34883e403 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:48.138427 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff2dbaa-73fa-491b-b047-dbe34883e403 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:48.220194 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 815/1641] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:15:47 2022] POST /identity/v3/users => generated 308 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:15:48.221322 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff2dbaa-73fa-491b-b047-dbe34883e403 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:48.284903 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff2dbaa-73fa-491b-b047-dbe34883e403 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:48.322102 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b9c8d9-7fa9-4148-b95f-a92791f1fcfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:48.324800 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b9c8d9-7fa9-4148-b95f-a92791f1fcfa admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:48.324800 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b9c8d9-7fa9-4148-b95f-a92791f1fcfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:48.325162 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b9c8d9-7fa9-4148-b95f-a92791f1fcfa admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:48.331059 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b9c8d9-7fa9-4148-b95f-a92791f1fcfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:48.333671 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b9c8d9-7fa9-4148-b95f-a92791f1fcfa admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:48.405922 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 816/1642] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:48 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:15:48.407255 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b9c8d9-7fa9-4148-b95f-a92791f1fcfa admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:48.455367 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b9c8d9-7fa9-4148-b95f-a92791f1fcfa admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:48.456607 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547a1787-18a3-4ad0-8570-83d428bf3976 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:48.474599 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60b8d1a-4225-4881-974d-0007ff35a366 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:48.475561 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60b8d1a-4225-4881-974d-0007ff35a366 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:48.475997 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60b8d1a-4225-4881-974d-0007ff35a366 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:48.476427 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60b8d1a-4225-4881-974d-0007ff35a366 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:48.484046 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7814841d-d884-49f1-a295-1c34da5849ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:48.485772 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7814841d-d884-49f1-a295-1c34da5849ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:48.487897 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7814841d-d884-49f1-a295-1c34da5849ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:48.488748 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7814841d-d884-49f1-a295-1c34da5849ea admin admin] PATH_INFO: `/v3/projects/c46c1dc4ac9041a19014b5d839286ccf/users/0ac5999ab57f494092c08b440c596a2e/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:48.547164 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7814841d-d884-49f1-a295-1c34da5849ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=c46c1dc4ac9041a19014b5d839286ccf, user_id=0ac5999ab57f494092c08b440c596a2e, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:48.561026 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7814841d-d884-49f1-a295-1c34da5849ea admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:48.605983 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 817/1643] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:15:48 2022] PUT /identity/v3/projects/c46c1dc4ac9041a19014b5d839286ccf/users/0ac5999ab57f494092c08b440c596a2e/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:48.612303 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7814841d-d884-49f1-a295-1c34da5849ea admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:48.714625 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60b8d1a-4225-4881-974d-0007ff35a366 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:48.721453 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60b8d1a-4225-4881-974d-0007ff35a366 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:48.754111 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 827/1644] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:15:48 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 643 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:15:48.839208 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7814841d-d884-49f1-a295-1c34da5849ea admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:48.848048 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8a4ef5-16dc-480d-883a-f7c5e4b08105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:48.849737 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a4ef5-16dc-480d-883a-f7c5e4b08105 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:48.850954 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a4ef5-16dc-480d-883a-f7c5e4b08105 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:48.851707 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a4ef5-16dc-480d-883a-f7c5e4b08105 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:48.854042 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8a4ef5-16dc-480d-883a-f7c5e4b08105 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:48.857913 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8a4ef5-16dc-480d-883a-f7c5e4b08105 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:48.879398 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 818/1645] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:48 2022] GET /identity/v3/roles => generated 1420 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:15:48.884476 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60b8d1a-4225-4881-974d-0007ff35a366 service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:48.943954 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60b8d1a-4225-4881-974d-0007ff35a366 service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:48.961276 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6540a64d-1339-4aea-bfc4-132f4ba451e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:48.962434 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6540a64d-1339-4aea-bfc4-132f4ba451e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:48.962921 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6540a64d-1339-4aea-bfc4-132f4ba451e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:48.963324 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6540a64d-1339-4aea-bfc4-132f4ba451e0 admin admin] PATH_INFO: `/v3/projects/c46c1dc4ac9041a19014b5d839286ccf/users/0ac5999ab57f494092c08b440c596a2e/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:48.970513 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6540a64d-1339-4aea-bfc4-132f4ba451e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c46c1dc4ac9041a19014b5d839286ccf, user_id=0ac5999ab57f494092c08b440c596a2e, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:48.981459 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7fe7f5-3f7a-41e0-ace0-b8a10e5b1ed2 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:48.982246 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7fe7f5-3f7a-41e0-ace0-b8a10e5b1ed2 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:48.982784 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7fe7f5-3f7a-41e0-ace0-b8a10e5b1ed2 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:48.983709 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6540a64d-1339-4aea-bfc4-132f4ba451e0 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:48.985362 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 819/1646] 10.222.0.43 () {54 vars in 962 bytes} [Tue Sep 6 16:15:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:15:49.004797 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651bde58-417b-453e-b9d1-76867ff1c206 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:49.005922 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651bde58-417b-453e-b9d1-76867ff1c206 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:49.006821 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651bde58-417b-453e-b9d1-76867ff1c206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:49.053827 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 828/1647] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:15:48 2022] PUT /identity/v3/projects/c46c1dc4ac9041a19014b5d839286ccf/users/0ac5999ab57f494092c08b440c596a2e/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:49.104267 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-651bde58-417b-453e-b9d1-76867ff1c206 None None] MFA Rules not processed for user `63de41f4348243998029f7d07aa5dc40`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:49.106478 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-651bde58-417b-453e-b9d1-76867ff1c206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:49.265378 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:49.359918 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd3bf71-6249-4fd4-91a1-a68afadb1e47 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:49.360298 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd3bf71-6249-4fd4-91a1-a68afadb1e47 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:49.360812 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd3bf71-6249-4fd4-91a1-a68afadb1e47 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:49.362957 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 829/1648] 10.222.0.43 () {54 vars in 912 bytes} [Tue Sep 6 16:15:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 16:15:49.368016 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 820/1649] 10.222.0.43 () {58 vars in 1059 bytes} [Tue Sep 6 16:15:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 365 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:49.371317 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0201026-c506-4084-97ab-c3e6063f1963 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:49.372447 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0201026-c506-4084-97ab-c3e6063f1963 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:49.372447 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0201026-c506-4084-97ab-c3e6063f1963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:49.423159 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-e0201026-c506-4084-97ab-c3e6063f1963 None None] MFA Rules not processed for user `63de41f4348243998029f7d07aa5dc40`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:49.424924 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e0201026-c506-4084-97ab-c3e6063f1963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:49.441653 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 830/1650] 10.222.0.43 () {58 vars in 1009 bytes} [Tue Sep 6 16:15:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:49.489797 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651bde58-417b-453e-b9d1-76867ff1c206 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:49.651328 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651bde58-417b-453e-b9d1-76867ff1c206 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:49.659311 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05feb8b-1947-4c2a-8f63-0a64adc106b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:49.660388 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05feb8b-1947-4c2a-8f63-0a64adc106b3 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:49.661023 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05feb8b-1947-4c2a-8f63-0a64adc106b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:49.661504 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05feb8b-1947-4c2a-8f63-0a64adc106b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:49.678095 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05feb8b-1947-4c2a-8f63-0a64adc106b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:49.679798 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05feb8b-1947-4c2a-8f63-0a64adc106b3 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:49.706709 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 821/1651] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:15:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:15:52.399007 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c823c-195e-4278-8f8d-82ce65668924 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:52.399703 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c823c-195e-4278-8f8d-82ce65668924 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:52.400431 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c823c-195e-4278-8f8d-82ce65668924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:52.485856 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-ef4c823c-195e-4278-8f8d-82ce65668924 None None] MFA Rules not processed for user `0ac5999ab57f494092c08b440c596a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:52.489861 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ef4c823c-195e-4278-8f8d-82ce65668924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:52.534277 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:52.660676 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 831/1652] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:15:52 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:55.113672 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57e330c-6e03-4947-82f4-90b51be7b3be None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:55.114279 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57e330c-6e03-4947-82f4-90b51be7b3be None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:55.114279 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57e330c-6e03-4947-82f4-90b51be7b3be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:55.171452 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-f57e330c-6e03-4947-82f4-90b51be7b3be None None] MFA Rules not processed for user `e225aec64aee47449542aecbec0ae561`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:55.174092 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f57e330c-6e03-4947-82f4-90b51be7b3be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:55.267077 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 822/1653] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:15:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:55.288248 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4c823c-195e-4278-8f8d-82ce65668924 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:55.306990 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4c823c-195e-4278-8f8d-82ce65668924 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:55.317981 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef8bae1-a1f7-44af-9de8-729d22a09b36 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63de41f4348243998029f7d07aa5dc40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:55.318764 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef8bae1-a1f7-44af-9de8-729d22a09b36 service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:55.319375 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef8bae1-a1f7-44af-9de8-729d22a09b36 service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:55.319832 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef8bae1-a1f7-44af-9de8-729d22a09b36 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:55.330706 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef8bae1-a1f7-44af-9de8-729d22a09b36 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:55.339072 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef8bae1-a1f7-44af-9de8-729d22a09b36 service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:55.355143 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 832/1654] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:15:55 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:15:56.030139 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c88ba6-ea13-4afb-8ce5-4dd20016fbe2 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:56.031520 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c88ba6-ea13-4afb-8ce5-4dd20016fbe2 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:56.032654 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c88ba6-ea13-4afb-8ce5-4dd20016fbe2 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:56.037221 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 823/1655] 10.222.0.43 () {54 vars in 921 bytes} [Tue Sep 6 16:15:56 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:15:56.100097 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364ec144-4d09-4add-b40e-734e3b557af8 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:56.102622 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364ec144-4d09-4add-b40e-734e3b557af8 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:56.103633 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364ec144-4d09-4add-b40e-734e3b557af8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:56.178274 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-364ec144-4d09-4add-b40e-734e3b557af8 None None] MFA Rules not processed for user `71c2912a44f04bd983b4d4112dbee734`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:56.191801 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-364ec144-4d09-4add-b40e-734e3b557af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:56.546970 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 833/1656] 10.222.0.43 () {58 vars in 1018 bytes} [Tue Sep 6 16:15:56 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 448 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:56.581979 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c88ba6-ea13-4afb-8ce5-4dd20016fbe2 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:56.625633 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c88ba6-ea13-4afb-8ce5-4dd20016fbe2 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:56.638501 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d044ebe2-2dff-41b9-8281-0ef7a66b5663 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71c2912a44f04bd983b4d4112dbee734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:56.639137 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d044ebe2-2dff-41b9-8281-0ef7a66b5663 service placement] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:56.639415 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d044ebe2-2dff-41b9-8281-0ef7a66b5663 service placement] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:56.639684 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d044ebe2-2dff-41b9-8281-0ef7a66b5663 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:56.663759 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d044ebe2-2dff-41b9-8281-0ef7a66b5663 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:56.666678 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d044ebe2-2dff-41b9-8281-0ef7a66b5663 service placement] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:56.684740 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 824/1657] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:15:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:15:56.997312 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb71ece-7ddb-4163-8025-0aa197048e36 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:56.998256 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb71ece-7ddb-4163-8025-0aa197048e36 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:56.998256 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb71ece-7ddb-4163-8025-0aa197048e36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:57.069821 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-dbb71ece-7ddb-4163-8025-0aa197048e36 None None] MFA Rules not processed for user `c3a65f26ab214feab60c7b6380539076`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:57.071250 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-dbb71ece-7ddb-4163-8025-0aa197048e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:57.230003 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 834/1658] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:15:56 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:57.245472 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d044ebe2-2dff-41b9-8281-0ef7a66b5663 service placement] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:57.263757 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d044ebe2-2dff-41b9-8281-0ef7a66b5663 service placement] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:57.279673 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9d60c9-98fc-460e-b3e6-6b2acbae380d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63de41f4348243998029f7d07aa5dc40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:57.280405 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9d60c9-98fc-460e-b3e6-6b2acbae380d service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:57.280937 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9d60c9-98fc-460e-b3e6-6b2acbae380d service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:57.281416 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9d60c9-98fc-460e-b3e6-6b2acbae380d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:57.302973 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9d60c9-98fc-460e-b3e6-6b2acbae380d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:57.305330 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9d60c9-98fc-460e-b3e6-6b2acbae380d service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:57.336255 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 825/1659] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:15:57 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:15:58.578330 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a08fec6-39c2-4fb3-87ba-aa1c9b103c35 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:58.578881 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a08fec6-39c2-4fb3-87ba-aa1c9b103c35 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:58.579057 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a08fec6-39c2-4fb3-87ba-aa1c9b103c35 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:58.580741 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 835/1660] 10.222.0.43 () {54 vars in 921 bytes} [Tue Sep 6 16:15:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:15:58.619474 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549eeb9-9b8e-4dcc-947a-26637c850df8 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:58.619961 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549eeb9-9b8e-4dcc-947a-26637c850df8 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:58.619961 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549eeb9-9b8e-4dcc-947a-26637c850df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:58.657165 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-5549eeb9-9b8e-4dcc-947a-26637c850df8 None None] MFA Rules not processed for user `71c2912a44f04bd983b4d4112dbee734`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:58.660221 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5549eeb9-9b8e-4dcc-947a-26637c850df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:58.685025 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 826/1661] 10.222.0.43 () {58 vars in 1018 bytes} [Tue Sep 6 16:15:58 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:58.723964 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a08fec6-39c2-4fb3-87ba-aa1c9b103c35 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:58.745217 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a08fec6-39c2-4fb3-87ba-aa1c9b103c35 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:58.752043 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a21f4-6665-4967-9c50-d96b59ca006f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:58.753277 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a21f4-6665-4967-9c50-d96b59ca006f service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:58.753559 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a21f4-6665-4967-9c50-d96b59ca006f service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:58.753830 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a21f4-6665-4967-9c50-d96b59ca006f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:58.805694 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5549eeb9-9b8e-4dcc-947a-26637c850df8 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:58.824624 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:58.914315 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:58.928793 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8a21f4-6665-4967-9c50-d96b59ca006f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:58.931476 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8a21f4-6665-4967-9c50-d96b59ca006f service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:58.978227 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 836/1662] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:15:58 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:15:58.979561 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a21f4-6665-4967-9c50-d96b59ca006f service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:58.987452 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5549eeb9-9b8e-4dcc-947a-26637c850df8 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:59.012414 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c7a5bf-8cd1-4740-b01f-bf333a08b973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:59.013263 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c7a5bf-8cd1-4740-b01f-bf333a08b973 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:59.013547 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c7a5bf-8cd1-4740-b01f-bf333a08b973 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:59.013973 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c7a5bf-8cd1-4740-b01f-bf333a08b973 admin admin] PATH_INFO: `/v3/users/9ecc6495a297447b9e0239a1c91a2f3d` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:59.015767 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c7a5bf-8cd1-4740-b01f-bf333a08b973 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ecc6495a297447b9e0239a1c91a2f3d)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:59.016174 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a21f4-6665-4967-9c50-d96b59ca006f service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:59.016854 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: WARNING py.warnings [None req-d1c7a5bf-8cd1-4740-b01f-bf333a08b973 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 16:15:59.016854 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: warnings.warn(msg) Sep 06 16:15:59.016854 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: Sep 06 16:15:59.019786 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c7a5bf-8cd1-4740-b01f-bf333a08b973 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:59.029498 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f32fbed-2b4c-4eb4-bfaa-f1ea821c33ae service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71c2912a44f04bd983b4d4112dbee734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:59.030160 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f32fbed-2b4c-4eb4-bfaa-f1ea821c33ae service placement] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:59.030440 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f32fbed-2b4c-4eb4-bfaa-f1ea821c33ae service placement] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:59.030707 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f32fbed-2b4c-4eb4-bfaa-f1ea821c33ae service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:59.054021 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f32fbed-2b4c-4eb4-bfaa-f1ea821c33ae service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:59.061044 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f32fbed-2b4c-4eb4-bfaa-f1ea821c33ae service placement] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:59.093079 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 837/1663] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:15:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:15:59.259271 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d1c7a5bf-8cd1-4740-b01f-bf333a08b973 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ecc6495a297447b9e0239a1c91a2f3d'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:59.259989 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d1c7a5bf-8cd1-4740-b01f-bf333a08b973 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ecc6495a297447b9e0239a1c91a2f3d'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:59.301606 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d1c7a5bf-8cd1-4740-b01f-bf333a08b973 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ecc6495a297447b9e0239a1c91a2f3d'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:59.397852 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d1c7a5bf-8cd1-4740-b01f-bf333a08b973 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ecc6495a297447b9e0239a1c91a2f3d'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:59.402294 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d1c7a5bf-8cd1-4740-b01f-bf333a08b973 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ecc6495a297447b9e0239a1c91a2f3d'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:59.466459 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 827/1664] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:15:58 2022] DELETE /identity/v3/users/9ecc6495a297447b9e0239a1c91a2f3d => generated 0 bytes in 658 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:15:59.715284 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f32fbed-2b4c-4eb4-bfaa-f1ea821c33ae service placement] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:59.734131 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f32fbed-2b4c-4eb4-bfaa-f1ea821c33ae service placement] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:59.897079 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2877611d-9c36-40c4-9953-206a9d5faa7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:59.907299 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2877611d-9c36-40c4-9953-206a9d5faa7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:59.909522 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2877611d-9c36-40c4-9953-206a9d5faa7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:59.910022 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2877611d-9c36-40c4-9953-206a9d5faa7a admin admin] PATH_INFO: `/v3/users/f180909caf24492db646b89ebd8ca214` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:59.914098 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2877611d-9c36-40c4-9953-206a9d5faa7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f180909caf24492db646b89ebd8ca214)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:59.921975 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2877611d-9c36-40c4-9953-206a9d5faa7a admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:00.082719 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-2877611d-9c36-40c4-9953-206a9d5faa7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f180909caf24492db646b89ebd8ca214'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:00.084354 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-2877611d-9c36-40c4-9953-206a9d5faa7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f180909caf24492db646b89ebd8ca214'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:00.102166 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-2877611d-9c36-40c4-9953-206a9d5faa7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f180909caf24492db646b89ebd8ca214'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:00.118605 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-2877611d-9c36-40c4-9953-206a9d5faa7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f180909caf24492db646b89ebd8ca214'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:00.135003 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-2877611d-9c36-40c4-9953-206a9d5faa7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f180909caf24492db646b89ebd8ca214'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:00.148313 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 838/1665] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:15:59 2022] DELETE /identity/v3/users/f180909caf24492db646b89ebd8ca214 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:00.149750 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c5f627-42ce-48bd-b1f4-66b404d62f1e None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:00.150647 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c5f627-42ce-48bd-b1f4-66b404d62f1e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:00.151134 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c5f627-42ce-48bd-b1f4-66b404d62f1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:00.195348 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-71c5f627-42ce-48bd-b1f4-66b404d62f1e None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:00.196947 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-71c5f627-42ce-48bd-b1f4-66b404d62f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:00.377487 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 828/1666] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:16:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:00.428794 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2877611d-9c36-40c4-9953-206a9d5faa7a admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:00.430739 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-2877611d-9c36-40c4-9953-206a9d5faa7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:00.544042 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:00.661184 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2877611d-9c36-40c4-9953-206a9d5faa7a admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:00.672027 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e07f186-a62a-4157-836e-3e3763b70681 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:00.675820 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e07f186-a62a-4157-836e-3e3763b70681 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:00.675820 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e07f186-a62a-4157-836e-3e3763b70681 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:00.677636 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e07f186-a62a-4157-836e-3e3763b70681 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:00.714837 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e07f186-a62a-4157-836e-3e3763b70681 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:00.716800 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e07f186-a62a-4157-836e-3e3763b70681 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:00.751844 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 839/1667] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:16:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:16:01.248293 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c5f627-42ce-48bd-b1f4-66b404d62f1e None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:01.253272 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-71c5f627-42ce-48bd-b1f4-66b404d62f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:01.283210 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e07f186-a62a-4157-836e-3e3763b70681 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:01.284644 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5e07f186-a62a-4157-836e-3e3763b70681 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:01.308830 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c5f627-42ce-48bd-b1f4-66b404d62f1e None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:01.312571 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e07f186-a62a-4157-836e-3e3763b70681 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:01.324318 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfdaa70-6a39-44e0-8ed0-0ddfae6c8699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:01.325024 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfdaa70-6a39-44e0-8ed0-0ddfae6c8699 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:01.326015 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53303126-be0e-47fa-bb28-059ec4d3f699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:01.326621 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfdaa70-6a39-44e0-8ed0-0ddfae6c8699 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:01.326912 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfdaa70-6a39-44e0-8ed0-0ddfae6c8699 admin admin] PATH_INFO: `/v3/projects/71de7da6100247d6a8defff682678e1f` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:01.329359 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53303126-be0e-47fa-bb28-059ec4d3f699 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:01.329791 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53303126-be0e-47fa-bb28-059ec4d3f699 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:01.330706 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfdaa70-6a39-44e0-8ed0-0ddfae6c8699 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71de7da6100247d6a8defff682678e1f)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:01.331197 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53303126-be0e-47fa-bb28-059ec4d3f699 admin admin] PATH_INFO: `/v3/projects/19286596a2414ae78ba7080b971364ca` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:01.334724 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfdaa70-6a39-44e0-8ed0-0ddfae6c8699 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:01.337271 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53303126-be0e-47fa-bb28-059ec4d3f699 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19286596a2414ae78ba7080b971364ca)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:01.338474 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: WARNING py.warnings [None req-53303126-be0e-47fa-bb28-059ec4d3f699 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 16:16:01.338474 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: warnings.warn(msg) Sep 06 16:16:01.338474 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: Sep 06 16:16:01.339974 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53303126-be0e-47fa-bb28-059ec4d3f699 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:01.433688 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-53303126-be0e-47fa-bb28-059ec4d3f699 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19286596a2414ae78ba7080b971364ca', 'request_id': 'req-53303126-be0e-47fa-bb28-059ec4d3f699'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:01.446509 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-53303126-be0e-47fa-bb28-059ec4d3f699 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19286596a2414ae78ba7080b971364ca', 'request_id': 'req-53303126-be0e-47fa-bb28-059ec4d3f699'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:01.460510 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-53303126-be0e-47fa-bb28-059ec4d3f699 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19286596a2414ae78ba7080b971364ca', 'request_id': 'req-53303126-be0e-47fa-bb28-059ec4d3f699'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:01.503208 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-cdfdaa70-6a39-44e0-8ed0-0ddfae6c8699 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71de7da6100247d6a8defff682678e1f', 'request_id': 'req-cdfdaa70-6a39-44e0-8ed0-0ddfae6c8699'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:01.507212 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:01.518083 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-53303126-be0e-47fa-bb28-059ec4d3f699 admin admin] The token cache is being invalidate because project 19286596a2414ae78ba7080b971364ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:16:01.519197 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-53303126-be0e-47fa-bb28-059ec4d3f699 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:01.520907 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-cdfdaa70-6a39-44e0-8ed0-0ddfae6c8699 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71de7da6100247d6a8defff682678e1f', 'request_id': 'req-cdfdaa70-6a39-44e0-8ed0-0ddfae6c8699'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:01.521739 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-53303126-be0e-47fa-bb28-059ec4d3f699 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:01.526334 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 840/1668] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:16:01 2022] DELETE /identity/v3/projects/19286596a2414ae78ba7080b971364ca => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:16:01.531884 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be453e7-8853-4b62-9160-08ce4d1165db None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:01.532280 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be453e7-8853-4b62-9160-08ce4d1165db None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:01.533275 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be453e7-8853-4b62-9160-08ce4d1165db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:01.552812 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-cdfdaa70-6a39-44e0-8ed0-0ddfae6c8699 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71de7da6100247d6a8defff682678e1f', 'request_id': 'req-cdfdaa70-6a39-44e0-8ed0-0ddfae6c8699'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:01.588540 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-cdfdaa70-6a39-44e0-8ed0-0ddfae6c8699 admin admin] The token cache is being invalidate because project 71de7da6100247d6a8defff682678e1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:16:01.588912 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-cdfdaa70-6a39-44e0-8ed0-0ddfae6c8699 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:01.592155 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-cdfdaa70-6a39-44e0-8ed0-0ddfae6c8699 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:01.596507 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 829/1669] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:16:01 2022] DELETE /identity/v3/projects/71de7da6100247d6a8defff682678e1f => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:01.601281 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfdaa70-6a39-44e0-8ed0-0ddfae6c8699 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:01.609374 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-cdfdaa70-6a39-44e0-8ed0-0ddfae6c8699 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:01.614507 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-0be453e7-8853-4b62-9160-08ce4d1165db None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:01.615740 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0be453e7-8853-4b62-9160-08ce4d1165db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:01.792479 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfdaa70-6a39-44e0-8ed0-0ddfae6c8699 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:01.797779 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f66652-068e-432d-b6a7-c6ec4d73c522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:01.798190 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 841/1670] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:16:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:01.798615 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f66652-068e-432d-b6a7-c6ec4d73c522 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:01.798702 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f66652-068e-432d-b6a7-c6ec4d73c522 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:01.798947 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f66652-068e-432d-b6a7-c6ec4d73c522 admin admin] PATH_INFO: `/v3/users/0ac5999ab57f494092c08b440c596a2e` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:01.814189 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db2fee6-8ac5-4c4f-8819-ba152e52bd30 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:01.814571 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db2fee6-8ac5-4c4f-8819-ba152e52bd30 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:01.814855 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db2fee6-8ac5-4c4f-8819-ba152e52bd30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:01.830219 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f66652-068e-432d-b6a7-c6ec4d73c522 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ac5999ab57f494092c08b440c596a2e)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:01.833346 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f66652-068e-432d-b6a7-c6ec4d73c522 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:01.859975 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-3db2fee6-8ac5-4c4f-8819-ba152e52bd30 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:01.861237 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-3db2fee6-8ac5-4c4f-8819-ba152e52bd30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:01.874303 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 842/1671] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:16:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:16:01.876236 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485c01a1-b59c-4120-89ed-9d1be44dd89d None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:01.876531 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485c01a1-b59c-4120-89ed-9d1be44dd89d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:01.876833 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485c01a1-b59c-4120-89ed-9d1be44dd89d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:01.915423 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-35f66652-068e-432d-b6a7-c6ec4d73c522 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ac5999ab57f494092c08b440c596a2e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:01.916397 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-35f66652-068e-432d-b6a7-c6ec4d73c522 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ac5999ab57f494092c08b440c596a2e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:01.926858 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-485c01a1-b59c-4120-89ed-9d1be44dd89d None None] MFA Rules not processed for user `8978e7cc78ab4c559dfd3e22cde34569`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:01.931679 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-485c01a1-b59c-4120-89ed-9d1be44dd89d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:01.933341 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-35f66652-068e-432d-b6a7-c6ec4d73c522 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ac5999ab57f494092c08b440c596a2e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:01.952587 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-35f66652-068e-432d-b6a7-c6ec4d73c522 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ac5999ab57f494092c08b440c596a2e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:01.954392 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-35f66652-068e-432d-b6a7-c6ec4d73c522 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ac5999ab57f494092c08b440c596a2e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:01.999208 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 830/1672] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:16:01 2022] DELETE /identity/v3/users/0ac5999ab57f494092c08b440c596a2e => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:02.000314 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f66652-068e-432d-b6a7-c6ec4d73c522 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:02.005255 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-35f66652-068e-432d-b6a7-c6ec4d73c522 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:02.047291 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:02.139264 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:02.162722 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 843/1673] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:16:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:02.164452 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab2d6a0-8766-4fa3-a6de-115bb51251cf None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:02.164739 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab2d6a0-8766-4fa3-a6de-115bb51251cf None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:02.165033 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab2d6a0-8766-4fa3-a6de-115bb51251cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:02.165280 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f66652-068e-432d-b6a7-c6ec4d73c522 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:02.173519 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18abd78a-8208-4b9c-bcf0-4bd0a3dc82f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:02.176147 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18abd78a-8208-4b9c-bcf0-4bd0a3dc82f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:02.176636 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18abd78a-8208-4b9c-bcf0-4bd0a3dc82f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:02.177162 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18abd78a-8208-4b9c-bcf0-4bd0a3dc82f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:02.178246 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18abd78a-8208-4b9c-bcf0-4bd0a3dc82f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:16:02.178918 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18abd78a-8208-4b9c-bcf0-4bd0a3dc82f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:02.181259 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18abd78a-8208-4b9c-bcf0-4bd0a3dc82f8 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:02.212588 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 831/1674] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 6 16:16:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:16:02.216832 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-0ab2d6a0-8766-4fa3-a6de-115bb51251cf None None] MFA Rules not processed for user `8978e7cc78ab4c559dfd3e22cde34569`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:02.217521 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18abd78a-8208-4b9c-bcf0-4bd0a3dc82f8 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:02.218426 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0ab2d6a0-8766-4fa3-a6de-115bb51251cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:02.243110 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18abd78a-8208-4b9c-bcf0-4bd0a3dc82f8 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:02.252436 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395c176e-33ff-435f-902a-b4f921476a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:02.254320 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395c176e-33ff-435f-902a-b4f921476a0c admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:02.254933 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395c176e-33ff-435f-902a-b4f921476a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:02.257188 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395c176e-33ff-435f-902a-b4f921476a0c admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:02.258298 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395c176e-33ff-435f-902a-b4f921476a0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:02.264324 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395c176e-33ff-435f-902a-b4f921476a0c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:02.325416 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 832/1675] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:16:02 2022] POST /identity/v3/projects => generated 367 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:16:02.327706 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4c2b16-b6ff-4a15-b9d4-0da9e41ce62a None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:02.328127 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4c2b16-b6ff-4a15-b9d4-0da9e41ce62a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:02.329101 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4c2b16-b6ff-4a15-b9d4-0da9e41ce62a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:02.343572 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 844/1676] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:16:02 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:02.347352 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab2d6a0-8766-4fa3-a6de-115bb51251cf None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:02.369636 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab2d6a0-8766-4fa3-a6de-115bb51251cf None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:02.373181 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-8d4c2b16-b6ff-4a15-b9d4-0da9e41ce62a None None] MFA Rules not processed for user `e225aec64aee47449542aecbec0ae561`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:02.374871 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-8d4c2b16-b6ff-4a15-b9d4-0da9e41ce62a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:02.446369 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7498397e-53b6-49e4-867e-79cb50c2faf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:02.448358 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7498397e-53b6-49e4-867e-79cb50c2faf7 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:02.448739 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7498397e-53b6-49e4-867e-79cb50c2faf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:02.449246 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7498397e-53b6-49e4-867e-79cb50c2faf7 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:02.449971 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7498397e-53b6-49e4-867e-79cb50c2faf7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:02.452266 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7498397e-53b6-49e4-867e-79cb50c2faf7 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:02.468635 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 833/1677] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:16:02 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:02.470439 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4c2b16-b6ff-4a15-b9d4-0da9e41ce62a None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:02.474447 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-8d4c2b16-b6ff-4a15-b9d4-0da9e41ce62a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:02.503277 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4c2b16-b6ff-4a15-b9d4-0da9e41ce62a None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:02.503501 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 845/1678] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:16:02 2022] POST /identity/v3/users => generated 284 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:16:02.504723 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7498397e-53b6-49e4-867e-79cb50c2faf7 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:02.506684 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7498397e-53b6-49e4-867e-79cb50c2faf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:02.512715 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110c0e3b-99b9-428b-8756-b94fb5705379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:02.513631 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110c0e3b-99b9-428b-8756-b94fb5705379 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:02.514062 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110c0e3b-99b9-428b-8756-b94fb5705379 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:02.514777 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110c0e3b-99b9-428b-8756-b94fb5705379 admin admin] PATH_INFO: `/v3/projects/c46c1dc4ac9041a19014b5d839286ccf` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:02.516202 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110c0e3b-99b9-428b-8756-b94fb5705379 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c46c1dc4ac9041a19014b5d839286ccf)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:02.520322 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110c0e3b-99b9-428b-8756-b94fb5705379 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:02.608490 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-110c0e3b-99b9-428b-8756-b94fb5705379 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c46c1dc4ac9041a19014b5d839286ccf', 'request_id': 'req-110c0e3b-99b9-428b-8756-b94fb5705379'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:02.615941 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-110c0e3b-99b9-428b-8756-b94fb5705379 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c46c1dc4ac9041a19014b5d839286ccf', 'request_id': 'req-110c0e3b-99b9-428b-8756-b94fb5705379'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:02.630966 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-110c0e3b-99b9-428b-8756-b94fb5705379 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c46c1dc4ac9041a19014b5d839286ccf', 'request_id': 'req-110c0e3b-99b9-428b-8756-b94fb5705379'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:02.631768 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7498397e-53b6-49e4-867e-79cb50c2faf7 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:02.640209 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:02.659453 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-110c0e3b-99b9-428b-8756-b94fb5705379 admin admin] The token cache is being invalidate because project c46c1dc4ac9041a19014b5d839286ccf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:16:02.659794 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-110c0e3b-99b9-428b-8756-b94fb5705379 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:02.660922 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-110c0e3b-99b9-428b-8756-b94fb5705379 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:02.663829 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 834/1679] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:16:02 2022] DELETE /identity/v3/projects/c46c1dc4ac9041a19014b5d839286ccf => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:02.666615 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110c0e3b-99b9-428b-8756-b94fb5705379 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:02.673015 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-110c0e3b-99b9-428b-8756-b94fb5705379 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:02.760328 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:02.765922 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110c0e3b-99b9-428b-8756-b94fb5705379 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:02.771663 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f162ec8-f2ce-4ba7-9f49-f99ea58b56de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:02.772524 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f162ec8-f2ce-4ba7-9f49-f99ea58b56de admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:02.773437 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f162ec8-f2ce-4ba7-9f49-f99ea58b56de admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:02.774400 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f162ec8-f2ce-4ba7-9f49-f99ea58b56de admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:02.774602 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f162ec8-f2ce-4ba7-9f49-f99ea58b56de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:02.776357 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f162ec8-f2ce-4ba7-9f49-f99ea58b56de admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:02.797090 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 835/1680] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:16:02 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:16:02.797090 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f26cfd3-802c-4b1f-894c-17432935a58b None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:02.797754 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f26cfd3-802c-4b1f-894c-17432935a58b None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:02.800105 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f26cfd3-802c-4b1f-894c-17432935a58b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:02.843815 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd31abd2-479d-4861-a8ec-1b9fb5220b1c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63de41f4348243998029f7d07aa5dc40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:02.847693 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd31abd2-479d-4861-a8ec-1b9fb5220b1c service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:02.848049 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd31abd2-479d-4861-a8ec-1b9fb5220b1c service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:02.848428 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd31abd2-479d-4861-a8ec-1b9fb5220b1c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:02.853308 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-bd31abd2-479d-4861-a8ec-1b9fb5220b1c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:02.854238 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-3f26cfd3-802c-4b1f-894c-17432935a58b None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:02.857031 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3f26cfd3-802c-4b1f-894c-17432935a58b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:02.876816 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 836/1681] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:16:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:02.878211 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f26cfd3-802c-4b1f-894c-17432935a58b None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:02.902395 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f26cfd3-802c-4b1f-894c-17432935a58b None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:02.914109 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ee9c8d-9e13-406e-a4b7-2a3082a33d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:02.914813 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ee9c8d-9e13-406e-a4b7-2a3082a33d99 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:02.915148 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ee9c8d-9e13-406e-a4b7-2a3082a33d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:02.915622 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ee9c8d-9e13-406e-a4b7-2a3082a33d99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:02.916294 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ee9c8d-9e13-406e-a4b7-2a3082a33d99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:02.923535 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ee9c8d-9e13-406e-a4b7-2a3082a33d99 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:02.965661 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 837/1682] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:16:02 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:16:02.974223 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc49d40-73e6-4395-8d03-53a0f6dbfdc3 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:02.974598 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc49d40-73e6-4395-8d03-53a0f6dbfdc3 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:02.975602 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc49d40-73e6-4395-8d03-53a0f6dbfdc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:03.016840 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd31abd2-479d-4861-a8ec-1b9fb5220b1c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:03.018502 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd31abd2-479d-4861-a8ec-1b9fb5220b1c service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:03.018611 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-0fc49d40-73e6-4395-8d03-53a0f6dbfdc3 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:03.019835 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0fc49d40-73e6-4395-8d03-53a0f6dbfdc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:03.046419 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 846/1683] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:16:02 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:16:03.048712 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd31abd2-479d-4861-a8ec-1b9fb5220b1c service nova] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:03.058921 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 838/1684] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:16:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:03.061359 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc49d40-73e6-4395-8d03-53a0f6dbfdc3 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:03.091142 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd31abd2-479d-4861-a8ec-1b9fb5220b1c service nova] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:03.100666 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc49d40-73e6-4395-8d03-53a0f6dbfdc3 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:03.100846 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc647547-d9d1-4b87-b5dc-73c89cc0f49b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:03.101528 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc647547-d9d1-4b87-b5dc-73c89cc0f49b admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:03.101805 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc647547-d9d1-4b87-b5dc-73c89cc0f49b admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:03.102074 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc647547-d9d1-4b87-b5dc-73c89cc0f49b admin admin] PATH_INFO: `/v3/projects/653424bb7be84b5e97908e142a4f52cc/users/aabc5f1f5e924a288ba38f640df57187/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:03.133197 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b680faee-691b-4b2d-903b-801528b282f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:03.133829 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b680faee-691b-4b2d-903b-801528b282f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:03.134109 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b680faee-691b-4b2d-903b-801528b282f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:03.134925 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b680faee-691b-4b2d-903b-801528b282f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:03.135033 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b680faee-691b-4b2d-903b-801528b282f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:16:03.135335 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b680faee-691b-4b2d-903b-801528b282f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:03.137033 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b680faee-691b-4b2d-903b-801528b282f6 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:03.151807 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc647547-d9d1-4b87-b5dc-73c89cc0f49b admin admin] RBAC: Authorizing `identity:create_grant(project_id=653424bb7be84b5e97908e142a4f52cc, user_id=aabc5f1f5e924a288ba38f640df57187, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:03.153484 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 839/1685] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 6 16:16:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:16:03.157776 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b680faee-691b-4b2d-903b-801528b282f6 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:03.159579 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc647547-d9d1-4b87-b5dc-73c89cc0f49b admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:03.173641 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b680faee-691b-4b2d-903b-801528b282f6 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:03.185092 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89730737-7c53-4010-bd50-58994e243b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:03.186338 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89730737-7c53-4010-bd50-58994e243b3a admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:03.186737 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89730737-7c53-4010-bd50-58994e243b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:03.187112 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89730737-7c53-4010-bd50-58994e243b3a admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:03.197054 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89730737-7c53-4010-bd50-58994e243b3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:03.200035 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 847/1686] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:16:03 2022] PUT /identity/v3/projects/653424bb7be84b5e97908e142a4f52cc/users/aabc5f1f5e924a288ba38f640df57187/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:03.211636 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77578881-4115-4d24-9e1a-1001cd0ede0d None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:03.211636 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77578881-4115-4d24-9e1a-1001cd0ede0d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:03.211636 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77578881-4115-4d24-9e1a-1001cd0ede0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:03.221461 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89730737-7c53-4010-bd50-58994e243b3a admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:03.283183 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-77578881-4115-4d24-9e1a-1001cd0ede0d None None] MFA Rules not processed for user `aabc5f1f5e924a288ba38f640df57187`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:03.284407 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-77578881-4115-4d24-9e1a-1001cd0ede0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:03.288619 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 840/1687] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:16:03 2022] POST /identity/v3/projects => generated 365 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:16:03.292513 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89730737-7c53-4010-bd50-58994e243b3a admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:03.442917 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 848/1688] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:16:03 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:03.448427 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89730737-7c53-4010-bd50-58994e243b3a admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:03.451408 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77578881-4115-4d24-9e1a-1001cd0ede0d None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:03.453119 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-77578881-4115-4d24-9e1a-1001cd0ede0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:03.465471 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5570f7-8da3-47d1-931b-0149610d8d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:03.466113 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5570f7-8da3-47d1-931b-0149610d8d21 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:03.466400 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5570f7-8da3-47d1-931b-0149610d8d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:03.466676 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5570f7-8da3-47d1-931b-0149610d8d21 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:03.467466 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5570f7-8da3-47d1-931b-0149610d8d21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:03.469542 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5570f7-8da3-47d1-931b-0149610d8d21 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:03.471077 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:03.531834 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 841/1689] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:16:03 2022] POST /identity/v3/users => generated 283 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:16:03.536267 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5570f7-8da3-47d1-931b-0149610d8d21 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:03.560121 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5570f7-8da3-47d1-931b-0149610d8d21 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:03.567087 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c529cf3-f846-40de-be2d-8049c469c17b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:03.567726 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c529cf3-f846-40de-be2d-8049c469c17b admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:03.568005 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c529cf3-f846-40de-be2d-8049c469c17b admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:03.568280 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c529cf3-f846-40de-be2d-8049c469c17b admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:03.568968 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c529cf3-f846-40de-be2d-8049c469c17b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:03.570693 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c529cf3-f846-40de-be2d-8049c469c17b admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:03.582463 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77578881-4115-4d24-9e1a-1001cd0ede0d None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:03.598738 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 842/1690] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:16:03 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:16:03.599155 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b7ad70-f7be-464d-8c17-b7fd7abc47e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:03.600434 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b7ad70-f7be-464d-8c17-b7fd7abc47e1 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:03.601113 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b7ad70-f7be-464d-8c17-b7fd7abc47e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:03.601564 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b7ad70-f7be-464d-8c17-b7fd7abc47e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:03.605505 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c529cf3-f846-40de-be2d-8049c469c17b admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:03.626609 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b7ad70-f7be-464d-8c17-b7fd7abc47e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:03.628455 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b7ad70-f7be-464d-8c17-b7fd7abc47e1 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:03.634250 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c529cf3-f846-40de-be2d-8049c469c17b admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:03.657143 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa5ef3e-3933-4333-88e5-18998a0a9baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:03.657788 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa5ef3e-3933-4333-88e5-18998a0a9baa admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:03.658505 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa5ef3e-3933-4333-88e5-18998a0a9baa admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:03.661083 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa5ef3e-3933-4333-88e5-18998a0a9baa admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:03.664290 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa5ef3e-3933-4333-88e5-18998a0a9baa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:03.665697 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 849/1691] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:16:03 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:16:03.667073 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f6b3e4-4a28-44bd-8cdf-99610aafdad6 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:03.667337 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f6b3e4-4a28-44bd-8cdf-99610aafdad6 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:03.668158 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa5ef3e-3933-4333-88e5-18998a0a9baa admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:03.669408 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f6b3e4-4a28-44bd-8cdf-99610aafdad6 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:03.670972 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 850/1692] 10.222.0.43 () {54 vars in 912 bytes} [Tue Sep 6 16:16:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:16:03.677964 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47da7061-589b-4007-a5ec-6fe0d2eb452d None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:03.678364 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47da7061-589b-4007-a5ec-6fe0d2eb452d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:03.678902 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47da7061-589b-4007-a5ec-6fe0d2eb452d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:03.690765 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 843/1693] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:16:03 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:16:03.693653 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa5ef3e-3933-4333-88e5-18998a0a9baa admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:03.708914 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa5ef3e-3933-4333-88e5-18998a0a9baa admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:03.714314 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28629ec-a313-4471-aad1-1fd59cd98bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:03.715130 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28629ec-a313-4471-aad1-1fd59cd98bd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:03.715513 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28629ec-a313-4471-aad1-1fd59cd98bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:03.715887 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28629ec-a313-4471-aad1-1fd59cd98bd5 admin admin] PATH_INFO: `/v3/projects/76f6f70455254ec196a6c9d6a3b27d22/users/b3d221141db24be8852d065f1da1f396/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:03.758804 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28629ec-a313-4471-aad1-1fd59cd98bd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76f6f70455254ec196a6c9d6a3b27d22, user_id=b3d221141db24be8852d065f1da1f396, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:03.764938 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28629ec-a313-4471-aad1-1fd59cd98bd5 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:03.772540 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-47da7061-589b-4007-a5ec-6fe0d2eb452d None None] MFA Rules not processed for user `10565aefeaaa40549ed38bb9e0f4b189`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:03.774049 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-47da7061-589b-4007-a5ec-6fe0d2eb452d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:03.786509 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 844/1694] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:16:03 2022] PUT /identity/v3/projects/76f6f70455254ec196a6c9d6a3b27d22/users/b3d221141db24be8852d065f1da1f396/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:16:03.796876 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3b1529-5a56-4196-8ef0-ff503978daed None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:03.797165 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3b1529-5a56-4196-8ef0-ff503978daed None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:03.797441 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3b1529-5a56-4196-8ef0-ff503978daed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:03.849812 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-ce3b1529-5a56-4196-8ef0-ff503978daed None None] MFA Rules not processed for user `b3d221141db24be8852d065f1da1f396`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:03.852068 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ce3b1529-5a56-4196-8ef0-ff503978daed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:03.891587 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 851/1695] 10.222.0.43 () {58 vars in 1009 bytes} [Tue Sep 6 16:16:03 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:03.895037 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:03.898573 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8b0415-ade3-4df0-bdb4-5d49799f8ed3 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:03.898863 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8b0415-ade3-4df0-bdb4-5d49799f8ed3 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:03.899273 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8b0415-ade3-4df0-bdb4-5d49799f8ed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:04.007816 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 845/1696] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:16:03 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:04.008998 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3b1529-5a56-4196-8ef0-ff503978daed None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:04.012989 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-2e8b0415-ade3-4df0-bdb4-5d49799f8ed3 None None] MFA Rules not processed for user `93096a58fcb34f2085a05ffa40df5937`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:04.015234 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ce3b1529-5a56-4196-8ef0-ff503978daed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:04.021596 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-2e8b0415-ade3-4df0-bdb4-5d49799f8ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:04.043464 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3b1529-5a56-4196-8ef0-ff503978daed None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:04.049921 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2a880d-ce27-4d13-8783-a2e7c446be18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:04.057981 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2a880d-ce27-4d13-8783-a2e7c446be18 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:04.058275 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2a880d-ce27-4d13-8783-a2e7c446be18 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:04.058582 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2a880d-ce27-4d13-8783-a2e7c446be18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:04.073782 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2a880d-ce27-4d13-8783-a2e7c446be18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:04.074230 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2a880d-ce27-4d13-8783-a2e7c446be18 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:04.084970 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 846/1697] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:16:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:16:04.092201 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2a880d-ce27-4d13-8783-a2e7c446be18 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:04.205883 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2a880d-ce27-4d13-8783-a2e7c446be18 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:04.213676 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba97642-aa45-431d-af4e-95845f218b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:04.214921 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba97642-aa45-431d-af4e-95845f218b91 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:04.215453 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba97642-aa45-431d-af4e-95845f218b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:04.216027 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba97642-aa45-431d-af4e-95845f218b91 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:04.217031 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba97642-aa45-431d-af4e-95845f218b91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:04.219291 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba97642-aa45-431d-af4e-95845f218b91 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:04.226864 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 852/1698] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:16:03 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:04.238210 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99de4f0f-80b3-4241-9954-a6103f1fd6d9 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:04.238509 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99de4f0f-80b3-4241-9954-a6103f1fd6d9 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:04.238760 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99de4f0f-80b3-4241-9954-a6103f1fd6d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:04.295692 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-99de4f0f-80b3-4241-9954-a6103f1fd6d9 None None] MFA Rules not processed for user `93096a58fcb34f2085a05ffa40df5937`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:04.298220 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 847/1699] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:16:04 2022] POST /identity/v3/projects => generated 361 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:16:04.305065 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba97642-aa45-431d-af4e-95845f218b91 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:04.308602 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-99de4f0f-80b3-4241-9954-a6103f1fd6d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:04.450636 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:04.508456 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 853/1700] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:16:04 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:16:04.517051 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba97642-aa45-431d-af4e-95845f218b91 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:04.522797 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f20fc6-28c6-4b2d-820b-8ffc19ebeb21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:04.523466 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f20fc6-28c6-4b2d-820b-8ffc19ebeb21 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:04.523820 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f20fc6-28c6-4b2d-820b-8ffc19ebeb21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:04.524160 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f20fc6-28c6-4b2d-820b-8ffc19ebeb21 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:04.525144 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f20fc6-28c6-4b2d-820b-8ffc19ebeb21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:04.527714 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f20fc6-28c6-4b2d-820b-8ffc19ebeb21 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:04.561307 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 848/1701] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:16:04 2022] POST /identity/v3/users => generated 287 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:16:04.561387 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99de4f0f-80b3-4241-9954-a6103f1fd6d9 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:04.578279 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99de4f0f-80b3-4241-9954-a6103f1fd6d9 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:04.585258 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d01aa2-98f2-43ab-9727-33b51cab31a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:04.586847 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d01aa2-98f2-43ab-9727-33b51cab31a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:04.587226 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d01aa2-98f2-43ab-9727-33b51cab31a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:04.587575 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d01aa2-98f2-43ab-9727-33b51cab31a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:04.588584 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d01aa2-98f2-43ab-9727-33b51cab31a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:04.591096 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d01aa2-98f2-43ab-9727-33b51cab31a2 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:04.608235 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 854/1702] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:16:04 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:16:04.610048 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f20fc6-28c6-4b2d-820b-8ffc19ebeb21 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:04.627020 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2434d6c2-6647-47b2-a1e3-b8b8d6be26f3 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:04.627319 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2434d6c2-6647-47b2-a1e3-b8b8d6be26f3 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:04.627567 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2434d6c2-6647-47b2-a1e3-b8b8d6be26f3 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:04.635652 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 855/1703] 10.222.0.43 () {54 vars in 912 bytes} [Tue Sep 6 16:16:04 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:16:04.639603 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f20fc6-28c6-4b2d-820b-8ffc19ebeb21 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:04.639732 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e437fd0-c09b-41f2-a02a-c247e823512b None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:04.639952 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e437fd0-c09b-41f2-a02a-c247e823512b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:04.640431 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e437fd0-c09b-41f2-a02a-c247e823512b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:04.657950 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f21b780-a3f6-43c4-a56f-f16e658bf3c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:04.658961 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f21b780-a3f6-43c4-a56f-f16e658bf3c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:04.659301 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f21b780-a3f6-43c4-a56f-f16e658bf3c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:04.659841 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f21b780-a3f6-43c4-a56f-f16e658bf3c6 admin admin] PATH_INFO: `/v3/projects/3bce525c5c3848c3802d504f50dbe337/users/b85e208e037144faa318f6046a9e7f4a/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:04.696319 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f21b780-a3f6-43c4-a56f-f16e658bf3c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bce525c5c3848c3802d504f50dbe337, user_id=b85e208e037144faa318f6046a9e7f4a, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:04.699056 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f21b780-a3f6-43c4-a56f-f16e658bf3c6 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:04.699340 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-8e437fd0-c09b-41f2-a02a-c247e823512b None None] MFA Rules not processed for user `10565aefeaaa40549ed38bb9e0f4b189`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:04.700697 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8e437fd0-c09b-41f2-a02a-c247e823512b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:04.725612 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 849/1704] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:16:04 2022] PUT /identity/v3/projects/3bce525c5c3848c3802d504f50dbe337/users/b85e208e037144faa318f6046a9e7f4a/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:16:04.730329 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f21b780-a3f6-43c4-a56f-f16e658bf3c6 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:04.752599 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:04.842401 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 856/1705] 10.222.0.43 () {58 vars in 1009 bytes} [Tue Sep 6 16:16:04 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:04.844240 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dafee70-326a-41b9-8401-8ab204ccfd4c None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:04.845226 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dafee70-326a-41b9-8401-8ab204ccfd4c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:04.845617 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dafee70-326a-41b9-8401-8ab204ccfd4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:04.847044 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f21b780-a3f6-43c4-a56f-f16e658bf3c6 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:04.854381 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91e5337-092d-4238-b8a2-4ea4b1044435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:04.855489 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91e5337-092d-4238-b8a2-4ea4b1044435 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:04.856125 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91e5337-092d-4238-b8a2-4ea4b1044435 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:04.857124 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91e5337-092d-4238-b8a2-4ea4b1044435 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:04.858771 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91e5337-092d-4238-b8a2-4ea4b1044435 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:04.862878 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91e5337-092d-4238-b8a2-4ea4b1044435 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:04.881944 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-4dafee70-326a-41b9-8401-8ab204ccfd4c None None] MFA Rules not processed for user `c3a65f26ab214feab60c7b6380539076`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:04.884019 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-4dafee70-326a-41b9-8401-8ab204ccfd4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:04.886554 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 850/1706] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:16:04 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:16:04.887639 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91e5337-092d-4238-b8a2-4ea4b1044435 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:04.890052 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b91e5337-092d-4238-b8a2-4ea4b1044435 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:05.040043 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 857/1707] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:16:04 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:05.040043 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dafee70-326a-41b9-8401-8ab204ccfd4c None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:05.054534 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91e5337-092d-4238-b8a2-4ea4b1044435 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:05.055470 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dafee70-326a-41b9-8401-8ab204ccfd4c None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:05.059068 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d6ff09-a55e-4532-8358-2d4bf89f3352 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:05.059775 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d6ff09-a55e-4532-8358-2d4bf89f3352 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:05.060074 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d6ff09-a55e-4532-8358-2d4bf89f3352 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:05.060335 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d6ff09-a55e-4532-8358-2d4bf89f3352 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:05.062361 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea20cb3-2e40-4fbd-a763-61caf4eec1c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:05.063123 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea20cb3-2e40-4fbd-a763-61caf4eec1c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:05.064106 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea20cb3-2e40-4fbd-a763-61caf4eec1c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:05.064509 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea20cb3-2e40-4fbd-a763-61caf4eec1c5 admin admin] PATH_INFO: `/v3/projects/3bce525c5c3848c3802d504f50dbe337/users/b85e208e037144faa318f6046a9e7f4a/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:05.067231 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea20cb3-2e40-4fbd-a763-61caf4eec1c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bce525c5c3848c3802d504f50dbe337, user_id=b85e208e037144faa318f6046a9e7f4a, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:05.071664 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea20cb3-2e40-4fbd-a763-61caf4eec1c5 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:05.097033 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d6ff09-a55e-4532-8358-2d4bf89f3352 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:05.098799 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d6ff09-a55e-4532-8358-2d4bf89f3352 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:05.113477 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 851/1708] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:16:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:16:05.115211 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d6ff09-a55e-4532-8358-2d4bf89f3352 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:05.117316 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-50d6ff09-a55e-4532-8358-2d4bf89f3352 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:05.130537 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 858/1709] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:16:05 2022] PUT /identity/v3/projects/3bce525c5c3848c3802d504f50dbe337/users/b85e208e037144faa318f6046a9e7f4a/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:05.141947 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59194b28-2eca-491b-8956-b471f2831b49 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:05.142516 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59194b28-2eca-491b-8956-b471f2831b49 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:05.142967 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59194b28-2eca-491b-8956-b471f2831b49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:05.184515 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-59194b28-2eca-491b-8956-b471f2831b49 None None] MFA Rules not processed for user `b85e208e037144faa318f6046a9e7f4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:05.186011 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-59194b28-2eca-491b-8956-b471f2831b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:05.241516 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d6ff09-a55e-4532-8358-2d4bf89f3352 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:05.327027 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:05.355447 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:05.361616 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 859/1710] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:16:05 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:05.363278 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59194b28-2eca-491b-8956-b471f2831b49 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:05.375101 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-59194b28-2eca-491b-8956-b471f2831b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:05.396074 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc996b0-b977-4396-9df9-8f1ac2092356 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63de41f4348243998029f7d07aa5dc40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:05.397169 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc996b0-b977-4396-9df9-8f1ac2092356 service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:05.397676 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc996b0-b977-4396-9df9-8f1ac2092356 service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:05.398197 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc996b0-b977-4396-9df9-8f1ac2092356 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:05.478860 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59194b28-2eca-491b-8956-b471f2831b49 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:05.487030 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc91b45-c738-48a1-b360-27d066932fae service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71c2912a44f04bd983b4d4112dbee734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:05.488118 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc91b45-c738-48a1-b360-27d066932fae service placement] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:05.488733 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc91b45-c738-48a1-b360-27d066932fae service placement] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:05.489231 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc91b45-c738-48a1-b360-27d066932fae service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:05.491277 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ffc91b45-c738-48a1-b360-27d066932fae service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:05.508422 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc91b45-c738-48a1-b360-27d066932fae service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:05.525110 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc91b45-c738-48a1-b360-27d066932fae service placement] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:05.576744 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 860/1711] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:16:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:16:05.577992 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc91b45-c738-48a1-b360-27d066932fae service placement] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:05.579626 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ffc91b45-c738-48a1-b360-27d066932fae service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:05.606455 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc996b0-b977-4396-9df9-8f1ac2092356 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:05.610823 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc996b0-b977-4396-9df9-8f1ac2092356 service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:05.645124 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 852/1712] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:16:05 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 531 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:16:05.650297 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc91b45-c738-48a1-b360-27d066932fae service placement] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:05.654140 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8e7dc6-1c7a-41a3-9631-a52a3a873f79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:05.654789 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8e7dc6-1c7a-41a3-9631-a52a3a873f79 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:05.655030 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8e7dc6-1c7a-41a3-9631-a52a3a873f79 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:05.655294 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8e7dc6-1c7a-41a3-9631-a52a3a873f79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:05.666889 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8e7dc6-1c7a-41a3-9631-a52a3a873f79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:05.668715 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8e7dc6-1c7a-41a3-9631-a52a3a873f79 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:05.680582 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 861/1713] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:16:05 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:16:05.875329 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc996b0-b977-4396-9df9-8f1ac2092356 service nova] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:05.889178 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc996b0-b977-4396-9df9-8f1ac2092356 service nova] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:05.901232 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3417cc30-033e-47c3-bbb4-998172f6a78b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:05.903919 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3417cc30-033e-47c3-bbb4-998172f6a78b service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:05.904397 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3417cc30-033e-47c3-bbb4-998172f6a78b service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:05.921990 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3417cc30-033e-47c3-bbb4-998172f6a78b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:06.159053 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3417cc30-033e-47c3-bbb4-998172f6a78b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:06.160952 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3417cc30-033e-47c3-bbb4-998172f6a78b service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:06.177261 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 853/1714] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:16:05 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:16:08.109581 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987aa23f-3d2d-4258-8096-b783e88b5ad3 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:08.110254 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987aa23f-3d2d-4258-8096-b783e88b5ad3 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:08.110254 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987aa23f-3d2d-4258-8096-b783e88b5ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:08.180694 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-987aa23f-3d2d-4258-8096-b783e88b5ad3 None None] MFA Rules not processed for user `93096a58fcb34f2085a05ffa40df5937`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:08.181960 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-987aa23f-3d2d-4258-8096-b783e88b5ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:08.232941 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 862/1715] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:16:08 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:11.626127 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5791186-a3f8-45f3-84e1-e528e345798e None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:11.626678 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5791186-a3f8-45f3-84e1-e528e345798e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:11.626876 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5791186-a3f8-45f3-84e1-e528e345798e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:11.788091 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-f5791186-a3f8-45f3-84e1-e528e345798e None None] MFA Rules not processed for user `d865006413b24ec3922619311f81f353`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:11.792775 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f5791186-a3f8-45f3-84e1-e528e345798e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:11.968396 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:12.048310 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 854/1716] 10.222.0.43 () {58 vars in 1009 bytes} [Tue Sep 6 16:16:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 423 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:12.303846 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987aa23f-3d2d-4258-8096-b783e88b5ad3 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:12.306981 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-987aa23f-3d2d-4258-8096-b783e88b5ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:12.604062 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987aa23f-3d2d-4258-8096-b783e88b5ad3 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:12.613242 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3069936b-f5d9-4614-9ee9-9eb31820dc06 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '312026c080b64aff9dd483d3591288a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:12.614636 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3069936b-f5d9-4614-9ee9-9eb31820dc06 service swift] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:12.614996 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3069936b-f5d9-4614-9ee9-9eb31820dc06 service swift] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:12.616155 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3069936b-f5d9-4614-9ee9-9eb31820dc06 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:12.635149 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:12.639424 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3069936b-f5d9-4614-9ee9-9eb31820dc06 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:12.645723 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3069936b-f5d9-4614-9ee9-9eb31820dc06 service swift] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:12.669305 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 863/1717] 10.222.0.43 () {60 vars in 1384 bytes} [Tue Sep 6 16:16:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 16:16:20.358651 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5791186-a3f8-45f3-84e1-e528e345798e None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:20.450545 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5791186-a3f8-45f3-84e1-e528e345798e None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:20.462007 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca23b0d-8b66-4fa4-ab36-e2ff66a8e162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:20.462669 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca23b0d-8b66-4fa4-ab36-e2ff66a8e162 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:20.463123 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca23b0d-8b66-4fa4-ab36-e2ff66a8e162 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:20.463473 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca23b0d-8b66-4fa4-ab36-e2ff66a8e162 admin admin] PATH_INFO: `/v3/users/b3d221141db24be8852d065f1da1f396` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:20.466876 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca23b0d-8b66-4fa4-ab36-e2ff66a8e162 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3d221141db24be8852d065f1da1f396)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:20.468941 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca23b0d-8b66-4fa4-ab36-e2ff66a8e162 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:20.570446 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-dca23b0d-8b66-4fa4-ab36-e2ff66a8e162 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3d221141db24be8852d065f1da1f396'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:20.571471 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-dca23b0d-8b66-4fa4-ab36-e2ff66a8e162 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3d221141db24be8852d065f1da1f396'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:20.592499 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-dca23b0d-8b66-4fa4-ab36-e2ff66a8e162 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3d221141db24be8852d065f1da1f396'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:20.627237 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-dca23b0d-8b66-4fa4-ab36-e2ff66a8e162 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3d221141db24be8852d065f1da1f396'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:20.628460 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-dca23b0d-8b66-4fa4-ab36-e2ff66a8e162 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3d221141db24be8852d065f1da1f396'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:20.655681 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 855/1718] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:16:20 2022] DELETE /identity/v3/users/b3d221141db24be8852d065f1da1f396 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:20.663713 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d88e0-2a51-4d1b-8659-e3d683602c7a None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:20.664302 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d88e0-2a51-4d1b-8659-e3d683602c7a None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:20.664984 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d88e0-2a51-4d1b-8659-e3d683602c7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:20.729574 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-d56d88e0-2a51-4d1b-8659-e3d683602c7a None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:20.731090 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d56d88e0-2a51-4d1b-8659-e3d683602c7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:20.921387 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 864/1719] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:16:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:20.945550 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca23b0d-8b66-4fa4-ab36-e2ff66a8e162 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:20.949145 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-dca23b0d-8b66-4fa4-ab36-e2ff66a8e162 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:21.047718 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca23b0d-8b66-4fa4-ab36-e2ff66a8e162 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:21.051881 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470600d7-8526-47b4-b5a4-1d76fa201728 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:21.052527 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470600d7-8526-47b4-b5a4-1d76fa201728 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:21.052928 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470600d7-8526-47b4-b5a4-1d76fa201728 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:21.053202 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470600d7-8526-47b4-b5a4-1d76fa201728 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:21.064246 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470600d7-8526-47b4-b5a4-1d76fa201728 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:21.065793 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470600d7-8526-47b4-b5a4-1d76fa201728 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:21.076415 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 856/1720] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:16:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:16:21.360990 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56d88e0-2a51-4d1b-8659-e3d683602c7a None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:21.362759 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d56d88e0-2a51-4d1b-8659-e3d683602c7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:21.381646 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56d88e0-2a51-4d1b-8659-e3d683602c7a None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:21.387383 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae47db43-f078-4d96-8761-d1177c747a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:21.388036 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae47db43-f078-4d96-8761-d1177c747a09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:21.388392 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae47db43-f078-4d96-8761-d1177c747a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:21.388640 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae47db43-f078-4d96-8761-d1177c747a09 admin admin] PATH_INFO: `/v3/projects/76f6f70455254ec196a6c9d6a3b27d22` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:21.389754 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae47db43-f078-4d96-8761-d1177c747a09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76f6f70455254ec196a6c9d6a3b27d22)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:21.391701 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae47db43-f078-4d96-8761-d1177c747a09 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:21.444820 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-ae47db43-f078-4d96-8761-d1177c747a09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76f6f70455254ec196a6c9d6a3b27d22', 'request_id': 'req-ae47db43-f078-4d96-8761-d1177c747a09'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:21.454304 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-ae47db43-f078-4d96-8761-d1177c747a09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76f6f70455254ec196a6c9d6a3b27d22', 'request_id': 'req-ae47db43-f078-4d96-8761-d1177c747a09'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:21.460271 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-ae47db43-f078-4d96-8761-d1177c747a09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76f6f70455254ec196a6c9d6a3b27d22', 'request_id': 'req-ae47db43-f078-4d96-8761-d1177c747a09'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:21.472612 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-ae47db43-f078-4d96-8761-d1177c747a09 admin admin] The token cache is being invalidate because project 76f6f70455254ec196a6c9d6a3b27d22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:16:21.472897 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-ae47db43-f078-4d96-8761-d1177c747a09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:21.473534 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-ae47db43-f078-4d96-8761-d1177c747a09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:21.475115 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 865/1721] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:16:21 2022] DELETE /identity/v3/projects/76f6f70455254ec196a6c9d6a3b27d22 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:21.479581 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470600d7-8526-47b4-b5a4-1d76fa201728 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:21.489881 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470600d7-8526-47b4-b5a4-1d76fa201728 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:21.498684 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:21.554314 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e047e324-60da-4211-8b7f-09b91b0ebe7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:21.554987 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e047e324-60da-4211-8b7f-09b91b0ebe7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:21.555265 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e047e324-60da-4211-8b7f-09b91b0ebe7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:21.555532 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e047e324-60da-4211-8b7f-09b91b0ebe7a admin admin] PATH_INFO: `/v3/users/b85e208e037144faa318f6046a9e7f4a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:21.556713 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e047e324-60da-4211-8b7f-09b91b0ebe7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b85e208e037144faa318f6046a9e7f4a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:21.558723 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e047e324-60da-4211-8b7f-09b91b0ebe7a admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:21.611518 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e047e324-60da-4211-8b7f-09b91b0ebe7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b85e208e037144faa318f6046a9e7f4a'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:21.612205 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e047e324-60da-4211-8b7f-09b91b0ebe7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b85e208e037144faa318f6046a9e7f4a'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:21.620827 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e047e324-60da-4211-8b7f-09b91b0ebe7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b85e208e037144faa318f6046a9e7f4a'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:21.635263 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e047e324-60da-4211-8b7f-09b91b0ebe7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b85e208e037144faa318f6046a9e7f4a'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:21.636143 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e047e324-60da-4211-8b7f-09b91b0ebe7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b85e208e037144faa318f6046a9e7f4a'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:21.652454 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 857/1722] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:16:21 2022] DELETE /identity/v3/users/b85e208e037144faa318f6046a9e7f4a => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:22.056926 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae47db43-f078-4d96-8761-d1177c747a09 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:22.058538 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ae47db43-f078-4d96-8761-d1177c747a09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:22.077845 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae47db43-f078-4d96-8761-d1177c747a09 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:22.185896 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c186a007-d721-4634-ade0-a9cb4e051dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:22.186730 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c186a007-d721-4634-ade0-a9cb4e051dde admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:22.187021 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c186a007-d721-4634-ade0-a9cb4e051dde admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:22.187285 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c186a007-d721-4634-ade0-a9cb4e051dde admin admin] PATH_INFO: `/v3/projects/3bce525c5c3848c3802d504f50dbe337` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:22.188988 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c186a007-d721-4634-ade0-a9cb4e051dde admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bce525c5c3848c3802d504f50dbe337)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:22.192325 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c186a007-d721-4634-ade0-a9cb4e051dde admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:22.228438 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:22.264926 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c186a007-d721-4634-ade0-a9cb4e051dde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bce525c5c3848c3802d504f50dbe337', 'request_id': 'req-c186a007-d721-4634-ade0-a9cb4e051dde'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:22.275319 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c186a007-d721-4634-ade0-a9cb4e051dde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bce525c5c3848c3802d504f50dbe337', 'request_id': 'req-c186a007-d721-4634-ade0-a9cb4e051dde'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:22.282405 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c186a007-d721-4634-ade0-a9cb4e051dde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bce525c5c3848c3802d504f50dbe337', 'request_id': 'req-c186a007-d721-4634-ade0-a9cb4e051dde'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:22.300827 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c186a007-d721-4634-ade0-a9cb4e051dde admin admin] The token cache is being invalidate because project 3bce525c5c3848c3802d504f50dbe337 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:16:22.301125 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c186a007-d721-4634-ade0-a9cb4e051dde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:22.301921 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c186a007-d721-4634-ade0-a9cb4e051dde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:22.303710 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 866/1723] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:16:22 2022] DELETE /identity/v3/projects/3bce525c5c3848c3802d504f50dbe337 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:22.308874 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b496d076-7560-43ec-b92e-105987935b7c None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:22.309318 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b496d076-7560-43ec-b92e-105987935b7c None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:22.309668 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b496d076-7560-43ec-b92e-105987935b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:22.330548 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-b496d076-7560-43ec-b92e-105987935b7c None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:22.331856 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b496d076-7560-43ec-b92e-105987935b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:22.407152 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 858/1724] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:16:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:22.415343 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38acd130-cf1a-43aa-bf16-6e8c320e1f58 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:22.415702 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38acd130-cf1a-43aa-bf16-6e8c320e1f58 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:22.415890 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38acd130-cf1a-43aa-bf16-6e8c320e1f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:22.436824 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-38acd130-cf1a-43aa-bf16-6e8c320e1f58 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:22.438400 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-38acd130-cf1a-43aa-bf16-6e8c320e1f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:22.446476 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 867/1725] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:16:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:22.450839 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b496d076-7560-43ec-b92e-105987935b7c None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:22.461015 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b496d076-7560-43ec-b92e-105987935b7c None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:22.465731 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3243edfd-8d34-40e4-9bdb-6b1f8c65122b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:22.466340 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3243edfd-8d34-40e4-9bdb-6b1f8c65122b admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:22.466621 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3243edfd-8d34-40e4-9bdb-6b1f8c65122b admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:22.466887 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3243edfd-8d34-40e4-9bdb-6b1f8c65122b admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:22.467562 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3243edfd-8d34-40e4-9bdb-6b1f8c65122b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:16:22.467860 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3243edfd-8d34-40e4-9bdb-6b1f8c65122b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:22.469548 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3243edfd-8d34-40e4-9bdb-6b1f8c65122b admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:22.486980 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 859/1726] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 6 16:16:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:16:22.488457 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38acd130-cf1a-43aa-bf16-6e8c320e1f58 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:22.503002 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38acd130-cf1a-43aa-bf16-6e8c320e1f58 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:22.508995 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ff959d-efba-4788-b227-022cbc91885a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:22.509593 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ff959d-efba-4788-b227-022cbc91885a admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:22.509874 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ff959d-efba-4788-b227-022cbc91885a admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:22.510138 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ff959d-efba-4788-b227-022cbc91885a admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:22.510761 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ff959d-efba-4788-b227-022cbc91885a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:22.512663 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ff959d-efba-4788-b227-022cbc91885a admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:22.550330 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 868/1727] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:16:22 2022] POST /identity/v3/projects => generated 383 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:16:22.551898 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3243edfd-8d34-40e4-9bdb-6b1f8c65122b admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:22.566752 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3243edfd-8d34-40e4-9bdb-6b1f8c65122b admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:22.585075 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:22.658926 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cb3d33-9969-4c3c-bc7f-9edcc91cf39f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:22.659913 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cb3d33-9969-4c3c-bc7f-9edcc91cf39f admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:22.660192 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cb3d33-9969-4c3c-bc7f-9edcc91cf39f admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:22.660462 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cb3d33-9969-4c3c-bc7f-9edcc91cf39f admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:22.661142 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cb3d33-9969-4c3c-bc7f-9edcc91cf39f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:22.663359 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cb3d33-9969-4c3c-bc7f-9edcc91cf39f admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:22.693141 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 860/1728] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:16:22 2022] POST /identity/v3/users => generated 292 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:16:22.694520 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ff959d-efba-4788-b227-022cbc91885a admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:22.705308 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ff959d-efba-4788-b227-022cbc91885a admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:22.710005 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be51f53-1afe-4644-81a4-9b7d3bb00ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:22.710674 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be51f53-1afe-4644-81a4-9b7d3bb00ed0 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:22.710951 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be51f53-1afe-4644-81a4-9b7d3bb00ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:22.711218 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be51f53-1afe-4644-81a4-9b7d3bb00ed0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:22.711868 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be51f53-1afe-4644-81a4-9b7d3bb00ed0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:22.713518 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be51f53-1afe-4644-81a4-9b7d3bb00ed0 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:22.728124 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 869/1729] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:16:22.729555 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cb3d33-9969-4c3c-bc7f-9edcc91cf39f admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:22.744394 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cb3d33-9969-4c3c-bc7f-9edcc91cf39f admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:22.749100 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb205e2-7388-458b-8d15-1a550eed1008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:22.749726 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb205e2-7388-458b-8d15-1a550eed1008 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:22.750029 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb205e2-7388-458b-8d15-1a550eed1008 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:22.750304 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb205e2-7388-458b-8d15-1a550eed1008 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:22.750967 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb205e2-7388-458b-8d15-1a550eed1008 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:22.752596 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb205e2-7388-458b-8d15-1a550eed1008 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:22.772807 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 861/1730] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:16:22.773068 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be51f53-1afe-4644-81a4-9b7d3bb00ed0 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:22.794879 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be51f53-1afe-4644-81a4-9b7d3bb00ed0 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:22.800771 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1c548d-f9b2-4061-95f0-23f180d7ad17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:22.801791 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1c548d-f9b2-4061-95f0-23f180d7ad17 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:22.802064 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1c548d-f9b2-4061-95f0-23f180d7ad17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:22.802347 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1c548d-f9b2-4061-95f0-23f180d7ad17 admin admin] PATH_INFO: `/v3/projects/2fc7ba1de9c0435aa594c9527c3c039d/users/229ddd51b1da4b989defa98c41aad4a6/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:22.821386 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1c548d-f9b2-4061-95f0-23f180d7ad17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fc7ba1de9c0435aa594c9527c3c039d, user_id=229ddd51b1da4b989defa98c41aad4a6, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:22.824103 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1c548d-f9b2-4061-95f0-23f180d7ad17 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:22.856736 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 870/1731] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:16:22 2022] PUT /identity/v3/projects/2fc7ba1de9c0435aa594c9527c3c039d/users/229ddd51b1da4b989defa98c41aad4a6/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:22.858466 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c060e-35cf-4a87-b246-b03e0d0bc76f None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:22.858466 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c060e-35cf-4a87-b246-b03e0d0bc76f None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:22.858680 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c060e-35cf-4a87-b246-b03e0d0bc76f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:22.894311 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-b45c060e-35cf-4a87-b246-b03e0d0bc76f None None] MFA Rules not processed for user `229ddd51b1da4b989defa98c41aad4a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:22.895550 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b45c060e-35cf-4a87-b246-b03e0d0bc76f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:22.994477 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 862/1732] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:16:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:16:22.999314 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1c548d-f9b2-4061-95f0-23f180d7ad17 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:23.091962 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1c548d-f9b2-4061-95f0-23f180d7ad17 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:23.110543 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58ece3f-db6a-41be-809a-a525301724c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:23.111431 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ece3f-db6a-41be-809a-a525301724c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:23.111708 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ece3f-db6a-41be-809a-a525301724c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:23.111977 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ece3f-db6a-41be-809a-a525301724c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:23.112681 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58ece3f-db6a-41be-809a-a525301724c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:23.114863 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58ece3f-db6a-41be-809a-a525301724c3 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:23.162165 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 871/1733] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:16:22 2022] POST /identity/v3/projects => generated 383 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:16:23.164029 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45c060e-35cf-4a87-b246-b03e0d0bc76f None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:23.176696 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45c060e-35cf-4a87-b246-b03e0d0bc76f None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:23.244402 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ddc629-d54f-4779-a83b-1f0254cca60e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:23.245060 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ddc629-d54f-4779-a83b-1f0254cca60e admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:23.245377 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ddc629-d54f-4779-a83b-1f0254cca60e admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:23.245635 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ddc629-d54f-4779-a83b-1f0254cca60e admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:23.246288 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ddc629-d54f-4779-a83b-1f0254cca60e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:23.248274 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ddc629-d54f-4779-a83b-1f0254cca60e admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:23.270037 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 863/1734] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:16:23 2022] POST /identity/v3/users => generated 298 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:16:23.274043 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58ece3f-db6a-41be-809a-a525301724c3 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:23.290445 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58ece3f-db6a-41be-809a-a525301724c3 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:23.295174 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37691875-6145-409b-bf23-b8bf2a7306f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:23.295807 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37691875-6145-409b-bf23-b8bf2a7306f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:23.296072 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37691875-6145-409b-bf23-b8bf2a7306f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:23.296341 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37691875-6145-409b-bf23-b8bf2a7306f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:23.300268 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37691875-6145-409b-bf23-b8bf2a7306f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:23.302617 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37691875-6145-409b-bf23-b8bf2a7306f9 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:23.315716 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 872/1735] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:16:23 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:16:23.320445 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ddc629-d54f-4779-a83b-1f0254cca60e admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:23.332145 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ddc629-d54f-4779-a83b-1f0254cca60e admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:23.338398 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb2a3f9-2fba-4cad-b6d9-19e1edee4350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:23.339076 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb2a3f9-2fba-4cad-b6d9-19e1edee4350 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:23.339387 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb2a3f9-2fba-4cad-b6d9-19e1edee4350 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:23.339660 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb2a3f9-2fba-4cad-b6d9-19e1edee4350 admin admin] PATH_INFO: `/v3/projects/bfd369464c4442e9bbb4ab02677340f5/users/0f46285ef85c4807a96b9645a1c532a2/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:23.355195 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:23.360490 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb2a3f9-2fba-4cad-b6d9-19e1edee4350 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfd369464c4442e9bbb4ab02677340f5, user_id=0f46285ef85c4807a96b9645a1c532a2, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:23.365185 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb2a3f9-2fba-4cad-b6d9-19e1edee4350 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:23.382369 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 864/1736] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:16:23 2022] PUT /identity/v3/projects/bfd369464c4442e9bbb4ab02677340f5/users/0f46285ef85c4807a96b9645a1c532a2/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:23.386422 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37691875-6145-409b-bf23-b8bf2a7306f9 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:23.400346 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37691875-6145-409b-bf23-b8bf2a7306f9 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:23.426437 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:23.489518 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b15a32-139d-4bea-8bb3-9811346ca371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:23.490165 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b15a32-139d-4bea-8bb3-9811346ca371 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:23.490441 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b15a32-139d-4bea-8bb3-9811346ca371 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:23.490712 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b15a32-139d-4bea-8bb3-9811346ca371 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:23.491435 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b15a32-139d-4bea-8bb3-9811346ca371 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:23.493317 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b15a32-139d-4bea-8bb3-9811346ca371 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:23.504191 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 873/1737] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:16:23 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:16:23.508880 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb2a3f9-2fba-4cad-b6d9-19e1edee4350 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:23.520151 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb2a3f9-2fba-4cad-b6d9-19e1edee4350 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:23.526032 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53b9b9d-2ea2-484d-adfd-509df1bf306a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:23.526770 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b9b9d-2ea2-484d-adfd-509df1bf306a admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:23.526991 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b9b9d-2ea2-484d-adfd-509df1bf306a admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:23.527262 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b9b9d-2ea2-484d-adfd-509df1bf306a admin admin] PATH_INFO: `/v3/projects/bfd369464c4442e9bbb4ab02677340f5/users/0f46285ef85c4807a96b9645a1c532a2/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:23.529706 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53b9b9d-2ea2-484d-adfd-509df1bf306a admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfd369464c4442e9bbb4ab02677340f5, user_id=0f46285ef85c4807a96b9645a1c532a2, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:23.532780 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53b9b9d-2ea2-484d-adfd-509df1bf306a admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:23.545362 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 865/1738] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:16:23 2022] PUT /identity/v3/projects/bfd369464c4442e9bbb4ab02677340f5/users/0f46285ef85c4807a96b9645a1c532a2/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:23.557086 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6798b76-8f25-4962-baf6-8bb7a0d6a5c1 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:23.557525 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6798b76-8f25-4962-baf6-8bb7a0d6a5c1 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:23.557846 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6798b76-8f25-4962-baf6-8bb7a0d6a5c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:23.590283 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-a6798b76-8f25-4962-baf6-8bb7a0d6a5c1 None None] MFA Rules not processed for user `0f46285ef85c4807a96b9645a1c532a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:23.591551 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a6798b76-8f25-4962-baf6-8bb7a0d6a5c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:23.669351 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 874/1739] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:16:23 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:23.683229 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53b9b9d-2ea2-484d-adfd-509df1bf306a admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:23.685124 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f53b9b9d-2ea2-484d-adfd-509df1bf306a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:23.750494 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53b9b9d-2ea2-484d-adfd-509df1bf306a admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:23.754739 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a85e06c-efa5-4ab9-a790-9167bfa9c26c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:23.755354 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a85e06c-efa5-4ab9-a790-9167bfa9c26c service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:23.755630 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a85e06c-efa5-4ab9-a790-9167bfa9c26c service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:23.755897 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a85e06c-efa5-4ab9-a790-9167bfa9c26c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:23.771823 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a85e06c-efa5-4ab9-a790-9167bfa9c26c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:23.773757 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a85e06c-efa5-4ab9-a790-9167bfa9c26c service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:23.784190 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 866/1740] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:16:23 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:16:24.590737 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6798b76-8f25-4962-baf6-8bb7a0d6a5c1 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:24.711827 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6798b76-8f25-4962-baf6-8bb7a0d6a5c1 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:24.732799 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2a290a-9602-410c-a028-d8d353907766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:24.733948 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2a290a-9602-410c-a028-d8d353907766 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:24.736328 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2a290a-9602-410c-a028-d8d353907766 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:24.736693 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2a290a-9602-410c-a028-d8d353907766 admin admin] PATH_INFO: `/v3/users/229ddd51b1da4b989defa98c41aad4a6` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:24.743436 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2a290a-9602-410c-a028-d8d353907766 admin admin] RBAC: Authorizing `identity:delete_user(user_id=229ddd51b1da4b989defa98c41aad4a6)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:24.745595 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2a290a-9602-410c-a028-d8d353907766 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:24.806615 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:24.809262 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-7f2a290a-9602-410c-a028-d8d353907766 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '229ddd51b1da4b989defa98c41aad4a6'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:24.810177 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-7f2a290a-9602-410c-a028-d8d353907766 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '229ddd51b1da4b989defa98c41aad4a6'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:24.818760 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-7f2a290a-9602-410c-a028-d8d353907766 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '229ddd51b1da4b989defa98c41aad4a6'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:24.831252 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-7f2a290a-9602-410c-a028-d8d353907766 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '229ddd51b1da4b989defa98c41aad4a6'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:24.840864 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-7f2a290a-9602-410c-a028-d8d353907766 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '229ddd51b1da4b989defa98c41aad4a6'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:24.846934 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 875/1741] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:16:24 2022] DELETE /identity/v3/users/229ddd51b1da4b989defa98c41aad4a6 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:24.851671 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58420aee-4148-4a74-919c-b59a1f9e1cec None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:24.852265 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58420aee-4148-4a74-919c-b59a1f9e1cec None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:24.852412 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58420aee-4148-4a74-919c-b59a1f9e1cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:24.872108 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-58420aee-4148-4a74-919c-b59a1f9e1cec None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:24.873347 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-58420aee-4148-4a74-919c-b59a1f9e1cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:24.938866 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 867/1742] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:16:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:24.949944 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2a290a-9602-410c-a028-d8d353907766 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:24.951448 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7f2a290a-9602-410c-a028-d8d353907766 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:25.025688 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2a290a-9602-410c-a028-d8d353907766 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:25.032792 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fb93ad-8d27-4345-b090-851fcd0ee8c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:25.033760 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fb93ad-8d27-4345-b090-851fcd0ee8c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:25.034411 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fb93ad-8d27-4345-b090-851fcd0ee8c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:25.035696 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fb93ad-8d27-4345-b090-851fcd0ee8c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:25.047090 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fb93ad-8d27-4345-b090-851fcd0ee8c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:25.048558 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fb93ad-8d27-4345-b090-851fcd0ee8c2 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:25.058574 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 876/1743] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:16:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:16:25.464503 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58420aee-4148-4a74-919c-b59a1f9e1cec None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:25.465967 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-58420aee-4148-4a74-919c-b59a1f9e1cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:25.479412 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58420aee-4148-4a74-919c-b59a1f9e1cec None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:25.483955 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5f3d99-a60d-4b67-9d11-d804ba82dadd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:25.484583 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f3d99-a60d-4b67-9d11-d804ba82dadd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:25.484895 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f3d99-a60d-4b67-9d11-d804ba82dadd admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:25.485161 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f3d99-a60d-4b67-9d11-d804ba82dadd admin admin] PATH_INFO: `/v3/projects/2fc7ba1de9c0435aa594c9527c3c039d` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:25.486257 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5f3d99-a60d-4b67-9d11-d804ba82dadd admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fc7ba1de9c0435aa594c9527c3c039d)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:25.488196 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5f3d99-a60d-4b67-9d11-d804ba82dadd admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:25.534391 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-1f5f3d99-a60d-4b67-9d11-d804ba82dadd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fc7ba1de9c0435aa594c9527c3c039d', 'request_id': 'req-1f5f3d99-a60d-4b67-9d11-d804ba82dadd'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:25.539795 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-1f5f3d99-a60d-4b67-9d11-d804ba82dadd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fc7ba1de9c0435aa594c9527c3c039d', 'request_id': 'req-1f5f3d99-a60d-4b67-9d11-d804ba82dadd'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:25.549710 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-1f5f3d99-a60d-4b67-9d11-d804ba82dadd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fc7ba1de9c0435aa594c9527c3c039d', 'request_id': 'req-1f5f3d99-a60d-4b67-9d11-d804ba82dadd'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:25.561063 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-1f5f3d99-a60d-4b67-9d11-d804ba82dadd admin admin] The token cache is being invalidate because project 2fc7ba1de9c0435aa594c9527c3c039d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:16:25.561236 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-1f5f3d99-a60d-4b67-9d11-d804ba82dadd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:25.561929 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-1f5f3d99-a60d-4b67-9d11-d804ba82dadd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:25.563826 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 868/1744] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:16:25 2022] DELETE /identity/v3/projects/2fc7ba1de9c0435aa594c9527c3c039d => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:16:25.568090 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fb93ad-8d27-4345-b090-851fcd0ee8c2 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:25.578736 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fb93ad-8d27-4345-b090-851fcd0ee8c2 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:25.637387 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd851dda-5c30-4e69-9f9d-2be9b896a7d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:25.638074 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd851dda-5c30-4e69-9f9d-2be9b896a7d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:25.638292 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd851dda-5c30-4e69-9f9d-2be9b896a7d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:25.638554 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd851dda-5c30-4e69-9f9d-2be9b896a7d6 admin admin] PATH_INFO: `/v3/users/0f46285ef85c4807a96b9645a1c532a2` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:25.639796 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd851dda-5c30-4e69-9f9d-2be9b896a7d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f46285ef85c4807a96b9645a1c532a2)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:25.641749 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd851dda-5c30-4e69-9f9d-2be9b896a7d6 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:25.689854 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-bd851dda-5c30-4e69-9f9d-2be9b896a7d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f46285ef85c4807a96b9645a1c532a2'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:25.690510 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-bd851dda-5c30-4e69-9f9d-2be9b896a7d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f46285ef85c4807a96b9645a1c532a2'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:25.697637 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-bd851dda-5c30-4e69-9f9d-2be9b896a7d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f46285ef85c4807a96b9645a1c532a2'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:25.705014 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:25.711720 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-bd851dda-5c30-4e69-9f9d-2be9b896a7d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f46285ef85c4807a96b9645a1c532a2'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:25.722186 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-bd851dda-5c30-4e69-9f9d-2be9b896a7d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f46285ef85c4807a96b9645a1c532a2'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:25.729755 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 877/1745] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:16:25 2022] DELETE /identity/v3/users/0f46285ef85c4807a96b9645a1c532a2 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:26.081656 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5f3d99-a60d-4b67-9d11-d804ba82dadd admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:26.083323 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1f5f3d99-a60d-4b67-9d11-d804ba82dadd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:26.093216 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:26.098153 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5f3d99-a60d-4b67-9d11-d804ba82dadd admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:26.190674 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21731048-1fc6-474d-916a-72de674c4ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:26.191488 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21731048-1fc6-474d-916a-72de674c4ed1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:26.191778 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21731048-1fc6-474d-916a-72de674c4ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:26.192176 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21731048-1fc6-474d-916a-72de674c4ed1 admin admin] PATH_INFO: `/v3/projects/bfd369464c4442e9bbb4ab02677340f5` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:26.193765 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21731048-1fc6-474d-916a-72de674c4ed1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfd369464c4442e9bbb4ab02677340f5)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:26.198982 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21731048-1fc6-474d-916a-72de674c4ed1 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:26.290228 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-21731048-1fc6-474d-916a-72de674c4ed1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfd369464c4442e9bbb4ab02677340f5', 'request_id': 'req-21731048-1fc6-474d-916a-72de674c4ed1'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:26.295989 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-21731048-1fc6-474d-916a-72de674c4ed1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfd369464c4442e9bbb4ab02677340f5', 'request_id': 'req-21731048-1fc6-474d-916a-72de674c4ed1'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:26.311598 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-21731048-1fc6-474d-916a-72de674c4ed1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfd369464c4442e9bbb4ab02677340f5', 'request_id': 'req-21731048-1fc6-474d-916a-72de674c4ed1'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:26.328782 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-21731048-1fc6-474d-916a-72de674c4ed1 admin admin] The token cache is being invalidate because project bfd369464c4442e9bbb4ab02677340f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:16:26.328984 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-21731048-1fc6-474d-916a-72de674c4ed1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:26.330086 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-21731048-1fc6-474d-916a-72de674c4ed1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:26.331537 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 869/1746] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:16:26 2022] DELETE /identity/v3/projects/bfd369464c4442e9bbb4ab02677340f5 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:26.338166 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf7928-b2db-4305-ad80-ec4020e2e7fb None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:26.338840 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf7928-b2db-4305-ad80-ec4020e2e7fb None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:26.339301 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf7928-b2db-4305-ad80-ec4020e2e7fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:26.379490 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-ddcf7928-b2db-4305-ad80-ec4020e2e7fb None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:26.381016 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ddcf7928-b2db-4305-ad80-ec4020e2e7fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:26.451136 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 878/1747] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:16:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:26.459228 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a27e76-0e88-480c-bc20-3fa561b1de4b None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:26.459507 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a27e76-0e88-480c-bc20-3fa561b1de4b None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:26.459835 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a27e76-0e88-480c-bc20-3fa561b1de4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:26.481695 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-97a27e76-0e88-480c-bc20-3fa561b1de4b None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:26.482893 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-97a27e76-0e88-480c-bc20-3fa561b1de4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:26.492805 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 870/1748] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:16:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:26.495386 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcf7928-b2db-4305-ad80-ec4020e2e7fb None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:26.505953 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcf7928-b2db-4305-ad80-ec4020e2e7fb None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:26.510671 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19da53b-0094-420e-b764-0c9b52ab01b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:26.511292 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19da53b-0094-420e-b764-0c9b52ab01b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:26.511570 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19da53b-0094-420e-b764-0c9b52ab01b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:26.511839 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19da53b-0094-420e-b764-0c9b52ab01b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:26.512497 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19da53b-0094-420e-b764-0c9b52ab01b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:16:26.512827 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19da53b-0094-420e-b764-0c9b52ab01b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:26.514444 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19da53b-0094-420e-b764-0c9b52ab01b0 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:26.526739 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 879/1749] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 6 16:16:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:16:26.531176 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a27e76-0e88-480c-bc20-3fa561b1de4b None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:26.541666 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a27e76-0e88-480c-bc20-3fa561b1de4b None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:26.547677 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c569aa2-a3a9-482d-918c-17de04b34b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:26.548268 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c569aa2-a3a9-482d-918c-17de04b34b47 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:26.548644 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c569aa2-a3a9-482d-918c-17de04b34b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:26.564861 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c569aa2-a3a9-482d-918c-17de04b34b47 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:26.564861 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c569aa2-a3a9-482d-918c-17de04b34b47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:26.564861 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c569aa2-a3a9-482d-918c-17de04b34b47 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:26.604884 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 871/1750] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:16:26 2022] POST /identity/v3/projects => generated 371 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:16:26.606467 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19da53b-0094-420e-b764-0c9b52ab01b0 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:26.617119 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19da53b-0094-420e-b764-0c9b52ab01b0 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:26.677305 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681dbee3-3d8b-4c2e-9ef4-14c6b2aab81b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:26.677969 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681dbee3-3d8b-4c2e-9ef4-14c6b2aab81b admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:26.678248 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681dbee3-3d8b-4c2e-9ef4-14c6b2aab81b admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:26.678576 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681dbee3-3d8b-4c2e-9ef4-14c6b2aab81b admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:26.679232 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681dbee3-3d8b-4c2e-9ef4-14c6b2aab81b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:26.681195 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681dbee3-3d8b-4c2e-9ef4-14c6b2aab81b admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:26.710570 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 880/1751] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:16:26 2022] POST /identity/v3/users => generated 286 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:16:26.716319 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c569aa2-a3a9-482d-918c-17de04b34b47 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:26.727920 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c569aa2-a3a9-482d-918c-17de04b34b47 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:26.733157 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7742d5ae-7323-45eb-89be-0e46a46cc958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:26.733811 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7742d5ae-7323-45eb-89be-0e46a46cc958 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:26.734084 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7742d5ae-7323-45eb-89be-0e46a46cc958 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:26.734354 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7742d5ae-7323-45eb-89be-0e46a46cc958 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:26.735007 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7742d5ae-7323-45eb-89be-0e46a46cc958 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:26.736648 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7742d5ae-7323-45eb-89be-0e46a46cc958 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:26.748166 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 872/1752] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:16:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:16:26.754298 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681dbee3-3d8b-4c2e-9ef4-14c6b2aab81b admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:26.770378 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681dbee3-3d8b-4c2e-9ef4-14c6b2aab81b admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:26.778211 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74e5791-3a22-463e-96aa-388229cc8ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:26.778845 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74e5791-3a22-463e-96aa-388229cc8ff2 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:26.779266 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74e5791-3a22-463e-96aa-388229cc8ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:26.779433 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74e5791-3a22-463e-96aa-388229cc8ff2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:26.780098 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74e5791-3a22-463e-96aa-388229cc8ff2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:26.781768 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74e5791-3a22-463e-96aa-388229cc8ff2 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:26.794552 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 881/1753] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:16:26 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:16:26.798595 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7742d5ae-7323-45eb-89be-0e46a46cc958 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:26.811295 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7742d5ae-7323-45eb-89be-0e46a46cc958 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:26.817607 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca2c84c-3375-48a5-912a-5159a688f9b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:26.818650 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca2c84c-3375-48a5-912a-5159a688f9b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:26.819064 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca2c84c-3375-48a5-912a-5159a688f9b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:26.819487 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca2c84c-3375-48a5-912a-5159a688f9b5 admin admin] PATH_INFO: `/v3/projects/e5df251fb443471694cba6a295b6c337/users/1a5951a7771c42e4bbae1aa56bded0bf/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:26.838714 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca2c84c-3375-48a5-912a-5159a688f9b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5df251fb443471694cba6a295b6c337, user_id=1a5951a7771c42e4bbae1aa56bded0bf, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:26.841466 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca2c84c-3375-48a5-912a-5159a688f9b5 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:26.861201 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 873/1754] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:16:26 2022] PUT /identity/v3/projects/e5df251fb443471694cba6a295b6c337/users/1a5951a7771c42e4bbae1aa56bded0bf/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:26.871475 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcb9f7e-c9e2-4111-9357-d1608d63fd05 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:26.871755 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcb9f7e-c9e2-4111-9357-d1608d63fd05 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:26.872031 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcb9f7e-c9e2-4111-9357-d1608d63fd05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:26.915171 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-0bcb9f7e-c9e2-4111-9357-d1608d63fd05 None None] MFA Rules not processed for user `1a5951a7771c42e4bbae1aa56bded0bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:26.916547 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0bcb9f7e-c9e2-4111-9357-d1608d63fd05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:26.956632 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:26.986552 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 882/1755] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:16:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:27.000004 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca2c84c-3375-48a5-912a-5159a688f9b5 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:27.001757 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5ca2c84c-3375-48a5-912a-5159a688f9b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:27.088676 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:27.172578 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca2c84c-3375-48a5-912a-5159a688f9b5 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:27.176425 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a8ade1-930a-410f-b582-3123d9889301 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c163b39e61648c385f065ce8091c3ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:27.177084 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a8ade1-930a-410f-b582-3123d9889301 service glance] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:27.178074 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a8ade1-930a-410f-b582-3123d9889301 service glance] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:27.178074 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a8ade1-930a-410f-b582-3123d9889301 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:27.191031 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a8ade1-930a-410f-b582-3123d9889301 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:27.193738 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a8ade1-930a-410f-b582-3123d9889301 service glance] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:27.205420 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 874/1756] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:16:26 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:16:28.732427 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a87baa1-44b8-4d2a-92e5-8dba7dc62f95 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:28.733100 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a87baa1-44b8-4d2a-92e5-8dba7dc62f95 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:28.733100 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a87baa1-44b8-4d2a-92e5-8dba7dc62f95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:28.762685 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-0a87baa1-44b8-4d2a-92e5-8dba7dc62f95 None None] MFA Rules not processed for user `10565aefeaaa40549ed38bb9e0f4b189`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:28.763922 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0a87baa1-44b8-4d2a-92e5-8dba7dc62f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:28.879658 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 883/1757] 10.222.0.64 () {58 vars in 1015 bytes} [Tue Sep 6 16:16:28 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:28.955184 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a8ade1-930a-410f-b582-3123d9889301 service glance] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:28.958370 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-92a8ade1-930a-410f-b582-3123d9889301 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:28.958908 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a87baa1-44b8-4d2a-92e5-8dba7dc62f95 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:28.960507 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0a87baa1-44b8-4d2a-92e5-8dba7dc62f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:28.983422 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a8ade1-930a-410f-b582-3123d9889301 service glance] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:28.983738 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a87baa1-44b8-4d2a-92e5-8dba7dc62f95 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:28.990143 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9d1ebc-a00a-448e-89ef-7cac3eeaa513 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:28.991304 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464815ac-f905-46cc-94f6-609dad8db70f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:28.991841 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d1ebc-a00a-448e-89ef-7cac3eeaa513 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:28.992132 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d1ebc-a00a-448e-89ef-7cac3eeaa513 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:28.993124 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d1ebc-a00a-448e-89ef-7cac3eeaa513 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:28.993708 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464815ac-f905-46cc-94f6-609dad8db70f service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:28.994205 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464815ac-f905-46cc-94f6-609dad8db70f service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:28.995181 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464815ac-f905-46cc-94f6-609dad8db70f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:29.011874 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9d1ebc-a00a-448e-89ef-7cac3eeaa513 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:29.013404 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9d1ebc-a00a-448e-89ef-7cac3eeaa513 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:29.027812 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 884/1758] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:16:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:16:29.028009 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464815ac-f905-46cc-94f6-609dad8db70f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:29.030545 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464815ac-f905-46cc-94f6-609dad8db70f service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:29.048143 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 875/1759] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:16:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:16:29.149330 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9d1ebc-a00a-448e-89ef-7cac3eeaa513 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:29.150998 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8e9d1ebc-a00a-448e-89ef-7cac3eeaa513 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:29.179152 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9d1ebc-a00a-448e-89ef-7cac3eeaa513 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:29.188369 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded5e14d-4f44-4ad8-a848-c6aabde2f5cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:29.189033 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded5e14d-4f44-4ad8-a848-c6aabde2f5cf service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:29.189312 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded5e14d-4f44-4ad8-a848-c6aabde2f5cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:29.189584 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded5e14d-4f44-4ad8-a848-c6aabde2f5cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:29.208390 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded5e14d-4f44-4ad8-a848-c6aabde2f5cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:29.211761 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded5e14d-4f44-4ad8-a848-c6aabde2f5cf service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:29.227509 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 885/1760] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:16:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:16:29.618839 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464815ac-f905-46cc-94f6-609dad8db70f service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:29.621132 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-464815ac-f905-46cc-94f6-609dad8db70f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:29.737547 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464815ac-f905-46cc-94f6-609dad8db70f service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:29.743329 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88404bc0-de16-4ed3-b1fc-e02d56adc7b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:29.744311 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88404bc0-de16-4ed3-b1fc-e02d56adc7b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:29.744803 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88404bc0-de16-4ed3-b1fc-e02d56adc7b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:29.745300 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88404bc0-de16-4ed3-b1fc-e02d56adc7b9 admin admin] PATH_INFO: `/v3/users/aabc5f1f5e924a288ba38f640df57187` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:29.746590 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88404bc0-de16-4ed3-b1fc-e02d56adc7b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aabc5f1f5e924a288ba38f640df57187)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:29.748679 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88404bc0-de16-4ed3-b1fc-e02d56adc7b9 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:29.827177 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-88404bc0-de16-4ed3-b1fc-e02d56adc7b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aabc5f1f5e924a288ba38f640df57187'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:29.828493 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-88404bc0-de16-4ed3-b1fc-e02d56adc7b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aabc5f1f5e924a288ba38f640df57187'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:29.842129 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-88404bc0-de16-4ed3-b1fc-e02d56adc7b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aabc5f1f5e924a288ba38f640df57187'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:29.872173 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-88404bc0-de16-4ed3-b1fc-e02d56adc7b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aabc5f1f5e924a288ba38f640df57187'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:29.872985 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-88404bc0-de16-4ed3-b1fc-e02d56adc7b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aabc5f1f5e924a288ba38f640df57187'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:29.891547 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 876/1761] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:16:29 2022] DELETE /identity/v3/users/aabc5f1f5e924a288ba38f640df57187 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:29.895812 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62559d63-54df-4b5d-9d7f-93b2d356daad None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:29.896795 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62559d63-54df-4b5d-9d7f-93b2d356daad None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:29.897344 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62559d63-54df-4b5d-9d7f-93b2d356daad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:29.922325 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-62559d63-54df-4b5d-9d7f-93b2d356daad None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:29.924376 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-62559d63-54df-4b5d-9d7f-93b2d356daad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:30.034323 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 886/1762] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:16:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:30.065917 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88404bc0-de16-4ed3-b1fc-e02d56adc7b9 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:30.071566 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-88404bc0-de16-4ed3-b1fc-e02d56adc7b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:30.111254 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:30.174951 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88404bc0-de16-4ed3-b1fc-e02d56adc7b9 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:30.179173 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d199a0d0-893b-43e9-aa5a-e4286c2b0dc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:30.179812 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d199a0d0-893b-43e9-aa5a-e4286c2b0dc9 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:30.180095 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d199a0d0-893b-43e9-aa5a-e4286c2b0dc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:30.180365 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d199a0d0-893b-43e9-aa5a-e4286c2b0dc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:30.191696 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d199a0d0-893b-43e9-aa5a-e4286c2b0dc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:30.193430 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d199a0d0-893b-43e9-aa5a-e4286c2b0dc9 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:30.207153 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 877/1763] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:16:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:16:30.661916 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08881388-8381-4a4e-825b-8b96df4d45a1 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:30.662539 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08881388-8381-4a4e-825b-8b96df4d45a1 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:30.662760 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08881388-8381-4a4e-825b-8b96df4d45a1 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:30.664527 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 887/1764] 10.222.0.43 () {54 vars in 921 bytes} [Tue Sep 6 16:16:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:16:30.686312 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a7071e-0419-48a0-a951-8739faf84062 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:30.686978 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a7071e-0419-48a0-a951-8739faf84062 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:30.687405 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a7071e-0419-48a0-a951-8739faf84062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:30.710610 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-79a7071e-0419-48a0-a951-8739faf84062 None None] MFA Rules not processed for user `63de41f4348243998029f7d07aa5dc40`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:30.712787 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-79a7071e-0419-48a0-a951-8739faf84062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:30.791035 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 878/1765] 10.222.0.43 () {58 vars in 1018 bytes} [Tue Sep 6 16:16:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:30.817856 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08881388-8381-4a4e-825b-8b96df4d45a1 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:30.820235 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-08881388-8381-4a4e-825b-8b96df4d45a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:30.866735 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08881388-8381-4a4e-825b-8b96df4d45a1 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:30.870923 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fb46e2-c542-4c12-8e7a-17fdc3b30a7b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63de41f4348243998029f7d07aa5dc40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:30.871845 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fb46e2-c542-4c12-8e7a-17fdc3b30a7b service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:30.872183 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fb46e2-c542-4c12-8e7a-17fdc3b30a7b service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:30.872579 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fb46e2-c542-4c12-8e7a-17fdc3b30a7b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:30.883726 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fb46e2-c542-4c12-8e7a-17fdc3b30a7b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:30.885582 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fb46e2-c542-4c12-8e7a-17fdc3b30a7b service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:30.897432 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 888/1766] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:16:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:16:31.271557 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a7071e-0419-48a0-a951-8739faf84062 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:31.274737 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-79a7071e-0419-48a0-a951-8739faf84062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:31.293468 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a7071e-0419-48a0-a951-8739faf84062 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:31.299221 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31abb6f-489d-44c4-b02f-381ae5534515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:31.300095 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31abb6f-489d-44c4-b02f-381ae5534515 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:31.300852 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31abb6f-489d-44c4-b02f-381ae5534515 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:31.301368 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31abb6f-489d-44c4-b02f-381ae5534515 admin admin] PATH_INFO: `/v3/projects/653424bb7be84b5e97908e142a4f52cc` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:31.302767 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31abb6f-489d-44c4-b02f-381ae5534515 admin admin] RBAC: Authorizing `identity:delete_project(project_id=653424bb7be84b5e97908e142a4f52cc)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:31.305675 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31abb6f-489d-44c4-b02f-381ae5534515 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:31.406578 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d31abb6f-489d-44c4-b02f-381ae5534515 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '653424bb7be84b5e97908e142a4f52cc', 'request_id': 'req-d31abb6f-489d-44c4-b02f-381ae5534515'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:31.417221 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d31abb6f-489d-44c4-b02f-381ae5534515 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '653424bb7be84b5e97908e142a4f52cc', 'request_id': 'req-d31abb6f-489d-44c4-b02f-381ae5534515'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:31.430602 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d31abb6f-489d-44c4-b02f-381ae5534515 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '653424bb7be84b5e97908e142a4f52cc', 'request_id': 'req-d31abb6f-489d-44c4-b02f-381ae5534515'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:31.446968 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d31abb6f-489d-44c4-b02f-381ae5534515 admin admin] The token cache is being invalidate because project 653424bb7be84b5e97908e142a4f52cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:16:31.447829 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d31abb6f-489d-44c4-b02f-381ae5534515 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:31.448390 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d31abb6f-489d-44c4-b02f-381ae5534515 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:31.450436 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 879/1767] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:16:31 2022] DELETE /identity/v3/projects/653424bb7be84b5e97908e142a4f52cc => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:31.457176 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c1facb-486c-4847-be57-8a8ecf8b786e None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:31.457698 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c1facb-486c-4847-be57-8a8ecf8b786e None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:31.459093 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c1facb-486c-4847-be57-8a8ecf8b786e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:31.517713 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-73c1facb-486c-4847-be57-8a8ecf8b786e None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:31.519569 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-73c1facb-486c-4847-be57-8a8ecf8b786e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:31.523727 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d984c7-5a14-49d8-a2f6-36c4c037fa2e None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:31.524083 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d984c7-5a14-49d8-a2f6-36c4c037fa2e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:31.524452 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d984c7-5a14-49d8-a2f6-36c4c037fa2e None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:31.526607 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 880/1768] 10.222.0.43 () {54 vars in 921 bytes} [Tue Sep 6 16:16:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:16:31.540728 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b967ad3b-e906-45f2-96ce-09c3fdfcd48b None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:31.541090 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b967ad3b-e906-45f2-96ce-09c3fdfcd48b None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:31.541433 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b967ad3b-e906-45f2-96ce-09c3fdfcd48b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:31.582991 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-b967ad3b-e906-45f2-96ce-09c3fdfcd48b None None] MFA Rules not processed for user `63de41f4348243998029f7d07aa5dc40`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:31.585165 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b967ad3b-e906-45f2-96ce-09c3fdfcd48b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:31.621497 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:31.682081 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 889/1769] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:16:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:31.690523 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:31.696428 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fc6c21-aa38-4e23-b3e4-6a0fa3c4da57 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:31.697919 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fc6c21-aa38-4e23-b3e4-6a0fa3c4da57 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:31.698576 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fc6c21-aa38-4e23-b3e4-6a0fa3c4da57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:31.762073 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-35fc6c21-aa38-4e23-b3e4-6a0fa3c4da57 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:31.762073 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-35fc6c21-aa38-4e23-b3e4-6a0fa3c4da57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:31.773915 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 881/1770] 10.222.0.43 () {58 vars in 1018 bytes} [Tue Sep 6 16:16:31 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:31.775309 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 890/1771] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:16:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:16:31.780245 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b967ad3b-e906-45f2-96ce-09c3fdfcd48b None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:31.794396 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fc6c21-aa38-4e23-b3e4-6a0fa3c4da57 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:31.796885 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-35fc6c21-aa38-4e23-b3e4-6a0fa3c4da57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:31.803378 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b967ad3b-e906-45f2-96ce-09c3fdfcd48b None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:31.810879 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129dbc25-66c4-4d22-97d6-cfb37a1436b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:31.811533 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129dbc25-66c4-4d22-97d6-cfb37a1436b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:31.811840 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129dbc25-66c4-4d22-97d6-cfb37a1436b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:31.812213 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129dbc25-66c4-4d22-97d6-cfb37a1436b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:31.814207 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129dbc25-66c4-4d22-97d6-cfb37a1436b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:16:31.814575 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129dbc25-66c4-4d22-97d6-cfb37a1436b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:31.817295 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fc6c21-aa38-4e23-b3e4-6a0fa3c4da57 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:31.817630 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129dbc25-66c4-4d22-97d6-cfb37a1436b1 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:31.833539 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0722638-04ab-4671-9ff0-e722cacd7800 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63de41f4348243998029f7d07aa5dc40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:31.833539 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0722638-04ab-4671-9ff0-e722cacd7800 service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:31.833539 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0722638-04ab-4671-9ff0-e722cacd7800 service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:31.833539 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0722638-04ab-4671-9ff0-e722cacd7800 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:31.837453 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0722638-04ab-4671-9ff0-e722cacd7800 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:31.839296 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0722638-04ab-4671-9ff0-e722cacd7800 service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:31.841592 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 882/1772] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 6 16:16:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:16:31.846124 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129dbc25-66c4-4d22-97d6-cfb37a1436b1 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:31.850540 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 891/1773] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:16:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:16:31.858588 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129dbc25-66c4-4d22-97d6-cfb37a1436b1 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:31.864195 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4de5fbb-4de7-4026-a254-51ef8efd4384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:31.865082 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de5fbb-4de7-4026-a254-51ef8efd4384 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:31.865429 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de5fbb-4de7-4026-a254-51ef8efd4384 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:31.865829 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de5fbb-4de7-4026-a254-51ef8efd4384 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:31.866527 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4de5fbb-4de7-4026-a254-51ef8efd4384 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:31.870811 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4de5fbb-4de7-4026-a254-51ef8efd4384 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:31.908793 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 883/1774] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:16:31 2022] POST /identity/v3/projects => generated 379 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:16:31.910229 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0722638-04ab-4671-9ff0-e722cacd7800 service nova] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:31.921734 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0722638-04ab-4671-9ff0-e722cacd7800 service nova] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:31.978866 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee0e186-c5c4-43fe-afb7-ea8a7ebcf580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:31.979473 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee0e186-c5c4-43fe-afb7-ea8a7ebcf580 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:31.979750 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee0e186-c5c4-43fe-afb7-ea8a7ebcf580 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:31.980017 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee0e186-c5c4-43fe-afb7-ea8a7ebcf580 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:31.980653 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee0e186-c5c4-43fe-afb7-ea8a7ebcf580 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:31.982645 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee0e186-c5c4-43fe-afb7-ea8a7ebcf580 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:32.004418 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 892/1775] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:16:31 2022] POST /identity/v3/users => generated 290 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:16:32.008728 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4de5fbb-4de7-4026-a254-51ef8efd4384 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:32.020300 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4de5fbb-4de7-4026-a254-51ef8efd4384 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:32.026138 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be73e66-97c0-4435-8715-366ae0e3e328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:32.026766 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be73e66-97c0-4435-8715-366ae0e3e328 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:32.027075 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be73e66-97c0-4435-8715-366ae0e3e328 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:32.027334 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be73e66-97c0-4435-8715-366ae0e3e328 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:32.027982 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be73e66-97c0-4435-8715-366ae0e3e328 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:32.029646 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be73e66-97c0-4435-8715-366ae0e3e328 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:32.040743 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 884/1776] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:16:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:16:32.045010 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee0e186-c5c4-43fe-afb7-ea8a7ebcf580 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:32.055937 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee0e186-c5c4-43fe-afb7-ea8a7ebcf580 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:32.060797 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac387d2f-95e8-40d8-a0f0-c0f5ff129bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:32.061431 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac387d2f-95e8-40d8-a0f0-c0f5ff129bb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:32.061710 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac387d2f-95e8-40d8-a0f0-c0f5ff129bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:32.062013 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac387d2f-95e8-40d8-a0f0-c0f5ff129bb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:32.062665 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac387d2f-95e8-40d8-a0f0-c0f5ff129bb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:32.064589 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac387d2f-95e8-40d8-a0f0-c0f5ff129bb9 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:32.075458 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 893/1777] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:16:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 16:16:32.080196 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be73e66-97c0-4435-8715-366ae0e3e328 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:32.091227 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be73e66-97c0-4435-8715-366ae0e3e328 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:32.096703 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2efe17-9ff7-427d-b926-60a46bddd3ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:32.097418 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2efe17-9ff7-427d-b926-60a46bddd3ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:32.097706 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2efe17-9ff7-427d-b926-60a46bddd3ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:32.097984 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2efe17-9ff7-427d-b926-60a46bddd3ea admin admin] PATH_INFO: `/v3/projects/1dcbb94c64c74add93bb685e9a88fccc/users/d1dee265493d48b5b435cee43436b059/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:32.115197 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2efe17-9ff7-427d-b926-60a46bddd3ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dcbb94c64c74add93bb685e9a88fccc, user_id=d1dee265493d48b5b435cee43436b059, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:32.118479 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2efe17-9ff7-427d-b926-60a46bddd3ea admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:32.132905 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 885/1778] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:16:32 2022] PUT /identity/v3/projects/1dcbb94c64c74add93bb685e9a88fccc/users/d1dee265493d48b5b435cee43436b059/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:32.144584 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce885ef-49eb-490d-a47c-0d9057a80cdc None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:32.144894 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce885ef-49eb-490d-a47c-0d9057a80cdc None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:32.145168 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce885ef-49eb-490d-a47c-0d9057a80cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:32.174491 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-5ce885ef-49eb-490d-a47c-0d9057a80cdc None None] MFA Rules not processed for user `d1dee265493d48b5b435cee43436b059`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:32.175762 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5ce885ef-49eb-490d-a47c-0d9057a80cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:32.243550 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 894/1779] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:16:32 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:16:32.248391 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2efe17-9ff7-427d-b926-60a46bddd3ea admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:32.354638 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2efe17-9ff7-427d-b926-60a46bddd3ea admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:32.375004 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680efba7-824b-4171-9a65-013345ccd716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:32.375662 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680efba7-824b-4171-9a65-013345ccd716 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:32.376005 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680efba7-824b-4171-9a65-013345ccd716 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:32.376334 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680efba7-824b-4171-9a65-013345ccd716 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:32.385560 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680efba7-824b-4171-9a65-013345ccd716 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:32.388047 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680efba7-824b-4171-9a65-013345ccd716 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:32.440686 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 886/1780] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:16:32 2022] POST /identity/v3/projects => generated 381 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:16:32.445498 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce885ef-49eb-490d-a47c-0d9057a80cdc None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:32.457162 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce885ef-49eb-490d-a47c-0d9057a80cdc None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:32.492656 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:32.518436 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37e49bc-7ef7-45e0-bada-00edf6193dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:32.519065 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37e49bc-7ef7-45e0-bada-00edf6193dd6 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:32.519488 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37e49bc-7ef7-45e0-bada-00edf6193dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:32.519749 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37e49bc-7ef7-45e0-bada-00edf6193dd6 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:32.520412 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37e49bc-7ef7-45e0-bada-00edf6193dd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:32.522501 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37e49bc-7ef7-45e0-bada-00edf6193dd6 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:32.551737 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 895/1781] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:16:32 2022] POST /identity/v3/users => generated 297 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:16:32.555426 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680efba7-824b-4171-9a65-013345ccd716 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:32.566779 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680efba7-824b-4171-9a65-013345ccd716 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:32.573025 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c2fa54-e9fb-46d2-b93f-38a1ed5c630b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:32.573646 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c2fa54-e9fb-46d2-b93f-38a1ed5c630b admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:32.574023 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c2fa54-e9fb-46d2-b93f-38a1ed5c630b admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:32.574183 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c2fa54-e9fb-46d2-b93f-38a1ed5c630b admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:32.574841 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c2fa54-e9fb-46d2-b93f-38a1ed5c630b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:32.576431 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c2fa54-e9fb-46d2-b93f-38a1ed5c630b admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:32.588327 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 887/1782] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:16:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:16:32.592689 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37e49bc-7ef7-45e0-bada-00edf6193dd6 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:32.603126 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37e49bc-7ef7-45e0-bada-00edf6193dd6 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:32.607817 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e7544f-47cd-44b8-931a-36a15cdf6fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:32.608389 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e7544f-47cd-44b8-931a-36a15cdf6fcd admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:32.608669 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e7544f-47cd-44b8-931a-36a15cdf6fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:32.608974 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e7544f-47cd-44b8-931a-36a15cdf6fcd admin admin] PATH_INFO: `/v3/projects/a9dd556cf1454779844d2564123f7615/users/34278f957cbd4cf4bdf88bd962a7c063/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:32.625395 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e7544f-47cd-44b8-931a-36a15cdf6fcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9dd556cf1454779844d2564123f7615, user_id=34278f957cbd4cf4bdf88bd962a7c063, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:32.628053 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e7544f-47cd-44b8-931a-36a15cdf6fcd admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:32.643467 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 896/1783] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:16:32 2022] PUT /identity/v3/projects/a9dd556cf1454779844d2564123f7615/users/34278f957cbd4cf4bdf88bd962a7c063/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:32.647597 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c2fa54-e9fb-46d2-b93f-38a1ed5c630b admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:32.657857 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c2fa54-e9fb-46d2-b93f-38a1ed5c630b admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:32.688452 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:32.718074 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de14f4c-5385-474f-aac9-81bbc96b5e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:32.718777 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de14f4c-5385-474f-aac9-81bbc96b5e0f admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:32.719058 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de14f4c-5385-474f-aac9-81bbc96b5e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:32.719327 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de14f4c-5385-474f-aac9-81bbc96b5e0f admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:32.719976 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de14f4c-5385-474f-aac9-81bbc96b5e0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:32.721639 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de14f4c-5385-474f-aac9-81bbc96b5e0f admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:32.732140 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 888/1784] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:16:32 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:16:32.736817 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e7544f-47cd-44b8-931a-36a15cdf6fcd admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:32.753112 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e7544f-47cd-44b8-931a-36a15cdf6fcd admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:32.757770 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc62211-7b57-4bc0-8418-cd63f0818668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:32.758437 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc62211-7b57-4bc0-8418-cd63f0818668 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:32.758712 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc62211-7b57-4bc0-8418-cd63f0818668 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:32.758979 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc62211-7b57-4bc0-8418-cd63f0818668 admin admin] PATH_INFO: `/v3/projects/a9dd556cf1454779844d2564123f7615/users/34278f957cbd4cf4bdf88bd962a7c063/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:32.760926 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc62211-7b57-4bc0-8418-cd63f0818668 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9dd556cf1454779844d2564123f7615, user_id=34278f957cbd4cf4bdf88bd962a7c063, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:32.763581 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc62211-7b57-4bc0-8418-cd63f0818668 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:32.776559 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 897/1785] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:16:32 2022] PUT /identity/v3/projects/a9dd556cf1454779844d2564123f7615/users/34278f957cbd4cf4bdf88bd962a7c063/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:32.786393 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c23c307-7f0d-4d03-8421-ac84aed8ab86 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:32.786798 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c23c307-7f0d-4d03-8421-ac84aed8ab86 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:32.786981 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c23c307-7f0d-4d03-8421-ac84aed8ab86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:32.817105 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-0c23c307-7f0d-4d03-8421-ac84aed8ab86 None None] MFA Rules not processed for user `34278f957cbd4cf4bdf88bd962a7c063`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:32.818310 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0c23c307-7f0d-4d03-8421-ac84aed8ab86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:32.901772 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 889/1786] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:16:32 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:32.915563 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc62211-7b57-4bc0-8418-cd63f0818668 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:32.917574 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0bc62211-7b57-4bc0-8418-cd63f0818668 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:32.995722 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc62211-7b57-4bc0-8418-cd63f0818668 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:33.000798 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341f1c27-619e-4f3b-8cad-6ad9258eb57b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:33.001687 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f1c27-619e-4f3b-8cad-6ad9258eb57b service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:33.001965 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f1c27-619e-4f3b-8cad-6ad9258eb57b service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:33.002281 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f1c27-619e-4f3b-8cad-6ad9258eb57b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:33.013570 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341f1c27-619e-4f3b-8cad-6ad9258eb57b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:33.015131 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341f1c27-619e-4f3b-8cad-6ad9258eb57b service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:33.028144 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 898/1787] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:16:32 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:16:33.117844 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c23c307-7f0d-4d03-8421-ac84aed8ab86 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:33.127830 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c23c307-7f0d-4d03-8421-ac84aed8ab86 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:33.132291 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd0a261-1869-4695-84b7-2f83dc847eb2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:33.133047 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd0a261-1869-4695-84b7-2f83dc847eb2 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:33.133327 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd0a261-1869-4695-84b7-2f83dc847eb2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:33.133596 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd0a261-1869-4695-84b7-2f83dc847eb2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:33.196401 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd0a261-1869-4695-84b7-2f83dc847eb2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:33.198001 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd0a261-1869-4695-84b7-2f83dc847eb2 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:33.207614 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 890/1788] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:16:33 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:16:43.022524 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=53,keystone:INSERT=5 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:43.203630 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=76 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:56.676801 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341f1c27-619e-4f3b-8cad-6ad9258eb57b service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:56.766198 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341f1c27-619e-4f3b-8cad-6ad9258eb57b service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:56.791583 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb74ba2-8b51-4f16-bbe8-86c14cc6b40a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:56.792489 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb74ba2-8b51-4f16-bbe8-86c14cc6b40a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:56.793056 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb74ba2-8b51-4f16-bbe8-86c14cc6b40a admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:56.793505 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb74ba2-8b51-4f16-bbe8-86c14cc6b40a admin admin] PATH_INFO: `/v3/users/d1dee265493d48b5b435cee43436b059` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:56.795149 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb74ba2-8b51-4f16-bbe8-86c14cc6b40a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1dee265493d48b5b435cee43436b059)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:56.797984 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb74ba2-8b51-4f16-bbe8-86c14cc6b40a admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:56.915154 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-dbb74ba2-8b51-4f16-bbe8-86c14cc6b40a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1dee265493d48b5b435cee43436b059'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:56.918377 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-dbb74ba2-8b51-4f16-bbe8-86c14cc6b40a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1dee265493d48b5b435cee43436b059'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:56.919405 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd0a261-1869-4695-84b7-2f83dc847eb2 service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:56.920857 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3dd0a261-1869-4695-84b7-2f83dc847eb2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:56.932447 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-dbb74ba2-8b51-4f16-bbe8-86c14cc6b40a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1dee265493d48b5b435cee43436b059'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:56.942998 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd0a261-1869-4695-84b7-2f83dc847eb2 service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:56.952468 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa73b330-d3e7-4555-bb92-0d13427fc3c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:56.953450 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa73b330-d3e7-4555-bb92-0d13427fc3c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:56.953887 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa73b330-d3e7-4555-bb92-0d13427fc3c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:56.954324 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa73b330-d3e7-4555-bb92-0d13427fc3c8 admin admin] PATH_INFO: `/v3/users/dc0dc02045fd49a69e4accdd626e01a7` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:56.954563 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-dbb74ba2-8b51-4f16-bbe8-86c14cc6b40a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1dee265493d48b5b435cee43436b059'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:56.956483 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa73b330-d3e7-4555-bb92-0d13427fc3c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc0dc02045fd49a69e4accdd626e01a7)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:56.960135 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa73b330-d3e7-4555-bb92-0d13427fc3c8 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:56.974491 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-dbb74ba2-8b51-4f16-bbe8-86c14cc6b40a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1dee265493d48b5b435cee43436b059'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:57.001918 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 899/1789] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:16:56 2022] DELETE /identity/v3/users/d1dee265493d48b5b435cee43436b059 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:57.007081 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7718e174-7932-4bcf-b1a0-a0f6a92c786b None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:57.007463 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7718e174-7932-4bcf-b1a0-a0f6a92c786b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:57.007864 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7718e174-7932-4bcf-b1a0-a0f6a92c786b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:57.025077 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-aa73b330-d3e7-4555-bb92-0d13427fc3c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc0dc02045fd49a69e4accdd626e01a7'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:57.025765 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-aa73b330-d3e7-4555-bb92-0d13427fc3c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc0dc02045fd49a69e4accdd626e01a7'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:57.035478 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-aa73b330-d3e7-4555-bb92-0d13427fc3c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc0dc02045fd49a69e4accdd626e01a7'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:57.038254 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-7718e174-7932-4bcf-b1a0-a0f6a92c786b None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:57.039492 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7718e174-7932-4bcf-b1a0-a0f6a92c786b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:57.054259 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-aa73b330-d3e7-4555-bb92-0d13427fc3c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc0dc02045fd49a69e4accdd626e01a7'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:57.055252 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-aa73b330-d3e7-4555-bb92-0d13427fc3c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc0dc02045fd49a69e4accdd626e01a7'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:57.079737 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 891/1790] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:16:56 2022] DELETE /identity/v3/users/dc0dc02045fd49a69e4accdd626e01a7 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:57.081622 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f9679-d7dc-485a-9ec8-f04372fcc73a None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:57.081910 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f9679-d7dc-485a-9ec8-f04372fcc73a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:57.082456 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f9679-d7dc-485a-9ec8-f04372fcc73a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:57.107292 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-179f9679-d7dc-485a-9ec8-f04372fcc73a None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:57.108583 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-179f9679-d7dc-485a-9ec8-f04372fcc73a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:57.138471 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 900/1791] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:16:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:57.154719 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7718e174-7932-4bcf-b1a0-a0f6a92c786b None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:57.156895 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7718e174-7932-4bcf-b1a0-a0f6a92c786b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:57.183684 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:57.202374 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 892/1792] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:16:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:57.216532 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179f9679-d7dc-485a-9ec8-f04372fcc73a None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:57.218196 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-179f9679-d7dc-485a-9ec8-f04372fcc73a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:57.232645 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7718e174-7932-4bcf-b1a0-a0f6a92c786b None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:57.294144 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179f9679-d7dc-485a-9ec8-f04372fcc73a None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:57.297999 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cde659d-7b4e-4b4b-bf55-f8217a0a07cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:57.298625 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cde659d-7b4e-4b4b-bf55-f8217a0a07cb service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:57.298932 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cde659d-7b4e-4b4b-bf55-f8217a0a07cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:57.299207 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cde659d-7b4e-4b4b-bf55-f8217a0a07cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:57.299602 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64e4a98-ecd1-436e-9be5-5d9728e96255 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:57.300537 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64e4a98-ecd1-436e-9be5-5d9728e96255 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:57.301003 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64e4a98-ecd1-436e-9be5-5d9728e96255 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:57.301421 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64e4a98-ecd1-436e-9be5-5d9728e96255 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:57.308881 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cde659d-7b4e-4b4b-bf55-f8217a0a07cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:57.310352 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cde659d-7b4e-4b4b-bf55-f8217a0a07cb service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:57.315962 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64e4a98-ecd1-436e-9be5-5d9728e96255 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:57.317761 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64e4a98-ecd1-436e-9be5-5d9728e96255 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:57.327059 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 901/1793] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:16:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:16:57.329646 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 893/1794] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:16:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:16:57.842985 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cde659d-7b4e-4b4b-bf55-f8217a0a07cb service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:57.847504 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8cde659d-7b4e-4b4b-bf55-f8217a0a07cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:57.848454 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64e4a98-ecd1-436e-9be5-5d9728e96255 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:57.851623 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-c64e4a98-ecd1-436e-9be5-5d9728e96255 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:57.886116 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64e4a98-ecd1-436e-9be5-5d9728e96255 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:57.891147 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cde659d-7b4e-4b4b-bf55-f8217a0a07cb service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:57.893086 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de15896f-50c7-433c-a89b-7e4d0ff1a4c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:57.893821 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de15896f-50c7-433c-a89b-7e4d0ff1a4c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:57.894356 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de15896f-50c7-433c-a89b-7e4d0ff1a4c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:57.894735 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de15896f-50c7-433c-a89b-7e4d0ff1a4c6 admin admin] PATH_INFO: `/v3/projects/1dcbb94c64c74add93bb685e9a88fccc` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:57.911353 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de15896f-50c7-433c-a89b-7e4d0ff1a4c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1dcbb94c64c74add93bb685e9a88fccc)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:57.917813 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de15896f-50c7-433c-a89b-7e4d0ff1a4c6 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:57.918953 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7431792-d2e6-4810-b227-d9172e4b8fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:57.921165 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7431792-d2e6-4810-b227-d9172e4b8fdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:57.921722 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7431792-d2e6-4810-b227-d9172e4b8fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:57.922409 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7431792-d2e6-4810-b227-d9172e4b8fdb admin admin] PATH_INFO: `/v3/projects/b1eddfa78b5745cb86be669ffeea8c97` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:57.925667 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7431792-d2e6-4810-b227-d9172e4b8fdb admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1eddfa78b5745cb86be669ffeea8c97)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:57.932128 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7431792-d2e6-4810-b227-d9172e4b8fdb admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:57.935798 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:58.005923 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-de15896f-50c7-433c-a89b-7e4d0ff1a4c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1dcbb94c64c74add93bb685e9a88fccc', 'request_id': 'req-de15896f-50c7-433c-a89b-7e4d0ff1a4c6'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:58.013004 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-de15896f-50c7-433c-a89b-7e4d0ff1a4c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1dcbb94c64c74add93bb685e9a88fccc', 'request_id': 'req-de15896f-50c7-433c-a89b-7e4d0ff1a4c6'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:58.026262 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-de15896f-50c7-433c-a89b-7e4d0ff1a4c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1dcbb94c64c74add93bb685e9a88fccc', 'request_id': 'req-de15896f-50c7-433c-a89b-7e4d0ff1a4c6'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:58.060733 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-de15896f-50c7-433c-a89b-7e4d0ff1a4c6 admin admin] The token cache is being invalidate because project 1dcbb94c64c74add93bb685e9a88fccc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:16:58.062013 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-de15896f-50c7-433c-a89b-7e4d0ff1a4c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:58.062977 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-de15896f-50c7-433c-a89b-7e4d0ff1a4c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:58.067923 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 894/1795] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:16:57 2022] DELETE /identity/v3/projects/1dcbb94c64c74add93bb685e9a88fccc => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:58.067923 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe677683-345e-4e52-9b08-f2506703b02f None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:58.067923 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe677683-345e-4e52-9b08-f2506703b02f None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:58.067923 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe677683-345e-4e52-9b08-f2506703b02f None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:58.068976 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 895/1796] 10.222.0.43 () {54 vars in 921 bytes} [Tue Sep 6 16:16:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:16:58.069879 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe677683-345e-4e52-9b08-f2506703b02f None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:58.071787 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-fe677683-345e-4e52-9b08-f2506703b02f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:58.083118 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e7431792-d2e6-4810-b227-d9172e4b8fdb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1eddfa78b5745cb86be669ffeea8c97', 'request_id': 'req-e7431792-d2e6-4810-b227-d9172e4b8fdb'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:58.111113 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e7431792-d2e6-4810-b227-d9172e4b8fdb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1eddfa78b5745cb86be669ffeea8c97', 'request_id': 'req-e7431792-d2e6-4810-b227-d9172e4b8fdb'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:58.126344 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e7431792-d2e6-4810-b227-d9172e4b8fdb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1eddfa78b5745cb86be669ffeea8c97', 'request_id': 'req-e7431792-d2e6-4810-b227-d9172e4b8fdb'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:58.146229 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e7431792-d2e6-4810-b227-d9172e4b8fdb admin admin] The token cache is being invalidate because project b1eddfa78b5745cb86be669ffeea8c97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:16:58.146684 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e7431792-d2e6-4810-b227-d9172e4b8fdb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:58.147516 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e7431792-d2e6-4810-b227-d9172e4b8fdb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:58.150277 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 902/1797] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:16:57 2022] DELETE /identity/v3/projects/b1eddfa78b5745cb86be669ffeea8c97 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:58.155188 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ee9936-6fae-435a-8d74-03ddf37d51b6 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:58.156064 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ee9936-6fae-435a-8d74-03ddf37d51b6 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:58.156419 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ee9936-6fae-435a-8d74-03ddf37d51b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:58.204845 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe677683-345e-4e52-9b08-f2506703b02f None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:58.210776 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab53967-78e6-4668-8259-85c0da4b4a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:58.211464 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab53967-78e6-4668-8259-85c0da4b4a61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:58.211776 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab53967-78e6-4668-8259-85c0da4b4a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:58.212080 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab53967-78e6-4668-8259-85c0da4b4a61 admin admin] PATH_INFO: `/v3/users/34278f957cbd4cf4bdf88bd962a7c063` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:58.213911 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab53967-78e6-4668-8259-85c0da4b4a61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34278f957cbd4cf4bdf88bd962a7c063)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:58.220880 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab53967-78e6-4668-8259-85c0da4b4a61 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:58.234180 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-90ee9936-6fae-435a-8d74-03ddf37d51b6 None None] MFA Rules not processed for user `63de41f4348243998029f7d07aa5dc40`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:58.235388 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-90ee9936-6fae-435a-8d74-03ddf37d51b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:58.315352 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2ab53967-78e6-4668-8259-85c0da4b4a61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34278f957cbd4cf4bdf88bd962a7c063'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:58.317036 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2ab53967-78e6-4668-8259-85c0da4b4a61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34278f957cbd4cf4bdf88bd962a7c063'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:58.331075 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2ab53967-78e6-4668-8259-85c0da4b4a61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34278f957cbd4cf4bdf88bd962a7c063'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:58.339658 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:58.349886 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2ab53967-78e6-4668-8259-85c0da4b4a61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34278f957cbd4cf4bdf88bd962a7c063'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:58.351162 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2ab53967-78e6-4668-8259-85c0da4b4a61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34278f957cbd4cf4bdf88bd962a7c063'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:58.360858 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 903/1798] 10.222.0.43 () {58 vars in 1018 bytes} [Tue Sep 6 16:16:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:58.362540 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ee9936-6fae-435a-8d74-03ddf37d51b6 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:58.365856 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-90ee9936-6fae-435a-8d74-03ddf37d51b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:58.371836 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 896/1799] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:16:58 2022] DELETE /identity/v3/users/34278f957cbd4cf4bdf88bd962a7c063 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:58.382094 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab53967-78e6-4668-8259-85c0da4b4a61 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:58.384448 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-2ab53967-78e6-4668-8259-85c0da4b4a61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:58.384694 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ee9936-6fae-435a-8d74-03ddf37d51b6 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:58.420628 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:58.491319 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f317819-81b3-4bb9-8928-1e36200b7925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:58.492559 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f317819-81b3-4bb9-8928-1e36200b7925 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:58.493125 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f317819-81b3-4bb9-8928-1e36200b7925 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:58.493663 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f317819-81b3-4bb9-8928-1e36200b7925 admin admin] PATH_INFO: `/v3/users/858442ac71bb43998f820c8476024e46` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:58.496781 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f317819-81b3-4bb9-8928-1e36200b7925 admin admin] RBAC: Authorizing `identity:delete_user(user_id=858442ac71bb43998f820c8476024e46)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:58.498969 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f317819-81b3-4bb9-8928-1e36200b7925 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:58.520325 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab53967-78e6-4668-8259-85c0da4b4a61 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:58.528801 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a099c8-ccd0-420b-a2d8-c9d3afd74229 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63de41f4348243998029f7d07aa5dc40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:58.529701 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a099c8-ccd0-420b-a2d8-c9d3afd74229 service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:58.529986 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a099c8-ccd0-420b-a2d8-c9d3afd74229 service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:58.530383 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a099c8-ccd0-420b-a2d8-c9d3afd74229 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:58.547036 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a099c8-ccd0-420b-a2d8-c9d3afd74229 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:58.549383 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a099c8-ccd0-420b-a2d8-c9d3afd74229 service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:58.563631 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 897/1800] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:16:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:16:58.572833 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8f317819-81b3-4bb9-8928-1e36200b7925 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '858442ac71bb43998f820c8476024e46'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:58.573543 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8f317819-81b3-4bb9-8928-1e36200b7925 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '858442ac71bb43998f820c8476024e46'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:58.587632 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8f317819-81b3-4bb9-8928-1e36200b7925 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '858442ac71bb43998f820c8476024e46'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:58.607272 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8f317819-81b3-4bb9-8928-1e36200b7925 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '858442ac71bb43998f820c8476024e46'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:58.618787 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8f317819-81b3-4bb9-8928-1e36200b7925 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '858442ac71bb43998f820c8476024e46'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:58.631589 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 904/1801] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:16:58 2022] DELETE /identity/v3/users/858442ac71bb43998f820c8476024e46 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:58.783562 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a099c8-ccd0-420b-a2d8-c9d3afd74229 service nova] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:58.785756 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-98a099c8-ccd0-420b-a2d8-c9d3afd74229 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:58.823815 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a099c8-ccd0-420b-a2d8-c9d3afd74229 service nova] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:58.888803 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d218a51-367e-45a5-a0c3-1e43ec4a7f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:58.889471 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d218a51-367e-45a5-a0c3-1e43ec4a7f35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:58.889759 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d218a51-367e-45a5-a0c3-1e43ec4a7f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:58.890199 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d218a51-367e-45a5-a0c3-1e43ec4a7f35 admin admin] PATH_INFO: `/v3/projects/a9dd556cf1454779844d2564123f7615` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:58.891357 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d218a51-367e-45a5-a0c3-1e43ec4a7f35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9dd556cf1454779844d2564123f7615)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:58.893373 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d218a51-367e-45a5-a0c3-1e43ec4a7f35 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:58.951564 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6d218a51-367e-45a5-a0c3-1e43ec4a7f35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9dd556cf1454779844d2564123f7615', 'request_id': 'req-6d218a51-367e-45a5-a0c3-1e43ec4a7f35'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:58.958950 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6d218a51-367e-45a5-a0c3-1e43ec4a7f35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9dd556cf1454779844d2564123f7615', 'request_id': 'req-6d218a51-367e-45a5-a0c3-1e43ec4a7f35'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:58.966520 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f317819-81b3-4bb9-8928-1e36200b7925 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:58.968916 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8f317819-81b3-4bb9-8928-1e36200b7925 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:58.972084 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6d218a51-367e-45a5-a0c3-1e43ec4a7f35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9dd556cf1454779844d2564123f7615', 'request_id': 'req-6d218a51-367e-45a5-a0c3-1e43ec4a7f35'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:58.995302 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6d218a51-367e-45a5-a0c3-1e43ec4a7f35 admin admin] The token cache is being invalidate because project a9dd556cf1454779844d2564123f7615 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:16:58.995594 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6d218a51-367e-45a5-a0c3-1e43ec4a7f35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:58.996727 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6d218a51-367e-45a5-a0c3-1e43ec4a7f35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:58.998814 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 898/1802] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:16:58 2022] DELETE /identity/v3/projects/a9dd556cf1454779844d2564123f7615 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:59.000430 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f317819-81b3-4bb9-8928-1e36200b7925 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:59.004083 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f68a72e-80b7-4126-97ab-8ee180be8a66 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:59.004454 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f68a72e-80b7-4126-97ab-8ee180be8a66 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:59.006396 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f68a72e-80b7-4126-97ab-8ee180be8a66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:59.038359 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-0f68a72e-80b7-4126-97ab-8ee180be8a66 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:59.040954 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0f68a72e-80b7-4126-97ab-8ee180be8a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:59.095456 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:59.104079 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a958de4c-5488-4a7d-98e6-812fce5cd67f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:59.105093 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a958de4c-5488-4a7d-98e6-812fce5cd67f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:59.105513 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a958de4c-5488-4a7d-98e6-812fce5cd67f admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:59.105948 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a958de4c-5488-4a7d-98e6-812fce5cd67f admin admin] PATH_INFO: `/v3/projects/2dfa02e2744a4e098f57ba1b9e75226a` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:59.107117 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a958de4c-5488-4a7d-98e6-812fce5cd67f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2dfa02e2744a4e098f57ba1b9e75226a)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:59.109422 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a958de4c-5488-4a7d-98e6-812fce5cd67f admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:59.131300 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:59.152938 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 899/1803] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:16:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:59.162752 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e79c00-1aba-4f2e-a0cf-0046740c1295 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:59.163482 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e79c00-1aba-4f2e-a0cf-0046740c1295 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:59.163976 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e79c00-1aba-4f2e-a0cf-0046740c1295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:59.173657 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a958de4c-5488-4a7d-98e6-812fce5cd67f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2dfa02e2744a4e098f57ba1b9e75226a', 'request_id': 'req-a958de4c-5488-4a7d-98e6-812fce5cd67f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:59.184529 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a958de4c-5488-4a7d-98e6-812fce5cd67f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2dfa02e2744a4e098f57ba1b9e75226a', 'request_id': 'req-a958de4c-5488-4a7d-98e6-812fce5cd67f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:59.188556 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-d6e79c00-1aba-4f2e-a0cf-0046740c1295 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:59.192829 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d6e79c00-1aba-4f2e-a0cf-0046740c1295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:59.196346 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a958de4c-5488-4a7d-98e6-812fce5cd67f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2dfa02e2744a4e098f57ba1b9e75226a', 'request_id': 'req-a958de4c-5488-4a7d-98e6-812fce5cd67f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:59.215072 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a958de4c-5488-4a7d-98e6-812fce5cd67f admin admin] The token cache is being invalidate because project 2dfa02e2744a4e098f57ba1b9e75226a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:16:59.215377 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a958de4c-5488-4a7d-98e6-812fce5cd67f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:59.216250 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a958de4c-5488-4a7d-98e6-812fce5cd67f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:59.222706 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 905/1804] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:16:58 2022] DELETE /identity/v3/projects/2dfa02e2744a4e098f57ba1b9e75226a => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:59.229854 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc8064f-88c7-410a-89d6-cbc308a190d7 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:59.231089 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc8064f-88c7-410a-89d6-cbc308a190d7 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:59.231772 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc8064f-88c7-410a-89d6-cbc308a190d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:59.261865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-8cc8064f-88c7-410a-89d6-cbc308a190d7 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:59.263262 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8cc8064f-88c7-410a-89d6-cbc308a190d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:59.285893 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 900/1805] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:16:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:16:59.290098 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e79c00-1aba-4f2e-a0cf-0046740c1295 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:59.304098 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e79c00-1aba-4f2e-a0cf-0046740c1295 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:59.310891 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559b0f83-56f7-459f-bada-1d8e423d28a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:59.311736 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b0f83-56f7-459f-bada-1d8e423d28a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:59.311980 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b0f83-56f7-459f-bada-1d8e423d28a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:59.312671 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b0f83-56f7-459f-bada-1d8e423d28a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:59.313444 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559b0f83-56f7-459f-bada-1d8e423d28a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:16:59.313812 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559b0f83-56f7-459f-bada-1d8e423d28a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:59.315792 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559b0f83-56f7-459f-bada-1d8e423d28a6 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:59.333599 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 901/1806] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 6 16:16:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:16:59.339838 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559b0f83-56f7-459f-bada-1d8e423d28a6 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:59.366328 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559b0f83-56f7-459f-bada-1d8e423d28a6 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:59.375007 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e67eaa5-65cc-46b1-8b09-be8b493b5c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:59.376833 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e67eaa5-65cc-46b1-8b09-be8b493b5c8f admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:59.377170 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e67eaa5-65cc-46b1-8b09-be8b493b5c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:59.377652 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e67eaa5-65cc-46b1-8b09-be8b493b5c8f admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:59.379677 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e67eaa5-65cc-46b1-8b09-be8b493b5c8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:59.390854 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e67eaa5-65cc-46b1-8b09-be8b493b5c8f admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:59.400231 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 906/1807] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:16:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:59.407105 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2db484a-d42f-4d60-9417-c1f6de602305 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:59.407386 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2db484a-d42f-4d60-9417-c1f6de602305 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:59.407828 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2db484a-d42f-4d60-9417-c1f6de602305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:59.442110 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-f2db484a-d42f-4d60-9417-c1f6de602305 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:59.443468 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f2db484a-d42f-4d60-9417-c1f6de602305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:59.450710 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 902/1808] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:16:59 2022] POST /identity/v3/projects => generated 383 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:16:59.455541 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e67eaa5-65cc-46b1-8b09-be8b493b5c8f admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:59.468695 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 907/1809] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:16:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:59.474392 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2db484a-d42f-4d60-9417-c1f6de602305 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:59.493069 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2db484a-d42f-4d60-9417-c1f6de602305 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:59.569762 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e67eaa5-65cc-46b1-8b09-be8b493b5c8f admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:59.575504 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8626156-804c-43ec-902c-4cbb30d2197f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:59.576183 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8626156-804c-43ec-902c-4cbb30d2197f admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:59.577098 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8626156-804c-43ec-902c-4cbb30d2197f admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:59.577487 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8626156-804c-43ec-902c-4cbb30d2197f admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:59.578486 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8626156-804c-43ec-902c-4cbb30d2197f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:59.579125 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2ba795-ad1f-46b5-b857-b98b12cf4087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:59.580307 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2ba795-ad1f-46b5-b857-b98b12cf4087 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:59.580846 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2ba795-ad1f-46b5-b857-b98b12cf4087 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:59.581351 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2ba795-ad1f-46b5-b857-b98b12cf4087 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:59.581539 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8626156-804c-43ec-902c-4cbb30d2197f admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:59.582311 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2ba795-ad1f-46b5-b857-b98b12cf4087 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:16:59.582727 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2ba795-ad1f-46b5-b857-b98b12cf4087 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:59.584676 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2ba795-ad1f-46b5-b857-b98b12cf4087 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:59.603063 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 908/1810] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 6 16:16:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:16:59.607288 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2ba795-ad1f-46b5-b857-b98b12cf4087 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:59.607630 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 903/1811] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:16:59 2022] POST /identity/v3/users => generated 292 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:16:59.611172 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8626156-804c-43ec-902c-4cbb30d2197f admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:59.620645 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2ba795-ad1f-46b5-b857-b98b12cf4087 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:59.622183 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8626156-804c-43ec-902c-4cbb30d2197f admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:59.625964 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc692efe-0a5f-44e2-9757-f7c79c6b9fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:59.626729 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc692efe-0a5f-44e2-9757-f7c79c6b9fad admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:59.627119 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc692efe-0a5f-44e2-9757-f7c79c6b9fad admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:59.627268 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a5b228-bd5c-44a7-98c6-c4c921b04bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:59.627599 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc692efe-0a5f-44e2-9757-f7c79c6b9fad admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:59.627862 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a5b228-bd5c-44a7-98c6-c4c921b04bda admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:59.628171 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a5b228-bd5c-44a7-98c6-c4c921b04bda admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:59.628546 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc692efe-0a5f-44e2-9757-f7c79c6b9fad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:59.628838 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a5b228-bd5c-44a7-98c6-c4c921b04bda admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:59.629859 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a5b228-bd5c-44a7-98c6-c4c921b04bda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:59.631137 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc692efe-0a5f-44e2-9757-f7c79c6b9fad admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:59.631810 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a5b228-bd5c-44a7-98c6-c4c921b04bda admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:59.647416 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 904/1812] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:16:59 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 16:16:59.651705 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a5b228-bd5c-44a7-98c6-c4c921b04bda admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:59.666198 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a5b228-bd5c-44a7-98c6-c4c921b04bda admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:59.671814 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 909/1813] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:16:59 2022] POST /identity/v3/projects => generated 385 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:16:59.673577 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc692efe-0a5f-44e2-9757-f7c79c6b9fad admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:59.688791 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:59.689538 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:59.769689 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc692efe-0a5f-44e2-9757-f7c79c6b9fad admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:59.782227 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30602896-2cff-4737-8618-6062faa217f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:59.784067 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30602896-2cff-4737-8618-6062faa217f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:59.785002 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30602896-2cff-4737-8618-6062faa217f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:59.785720 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30602896-2cff-4737-8618-6062faa217f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:59.787086 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30602896-2cff-4737-8618-6062faa217f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:59.791573 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30602896-2cff-4737-8618-6062faa217f2 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:59.791844 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08cbdea-a372-45c9-a578-ac8a06261569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:59.792605 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08cbdea-a372-45c9-a578-ac8a06261569 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:59.792982 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08cbdea-a372-45c9-a578-ac8a06261569 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:59.793322 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08cbdea-a372-45c9-a578-ac8a06261569 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:59.794086 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08cbdea-a372-45c9-a578-ac8a06261569 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:59.796993 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08cbdea-a372-45c9-a578-ac8a06261569 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:59.816319 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 905/1814] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:16:59 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:16:59.818587 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08cbdea-a372-45c9-a578-ac8a06261569 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:59.832098 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08cbdea-a372-45c9-a578-ac8a06261569 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:59.836461 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 910/1815] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:16:59 2022] POST /identity/v3/users => generated 293 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:16:59.838132 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68041d6a-e9c0-4306-899e-b7235ab461a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:59.838828 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68041d6a-e9c0-4306-899e-b7235ab461a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:59.839103 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68041d6a-e9c0-4306-899e-b7235ab461a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:59.839373 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68041d6a-e9c0-4306-899e-b7235ab461a0 admin admin] PATH_INFO: `/v3/projects/e3abc7d1ffc140cea2fd3e4afc677285/users/7acbd98fed99463ba2f672c7619d416b/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:59.842041 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30602896-2cff-4737-8618-6062faa217f2 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:59.863525 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68041d6a-e9c0-4306-899e-b7235ab461a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3abc7d1ffc140cea2fd3e4afc677285, user_id=7acbd98fed99463ba2f672c7619d416b, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:59.864306 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30602896-2cff-4737-8618-6062faa217f2 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:59.867164 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68041d6a-e9c0-4306-899e-b7235ab461a0 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:59.874001 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ccf118-3bfa-45d3-b34e-2584cde3a0c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:59.875064 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ccf118-3bfa-45d3-b34e-2584cde3a0c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:59.875426 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ccf118-3bfa-45d3-b34e-2584cde3a0c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:59.875678 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ccf118-3bfa-45d3-b34e-2584cde3a0c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:59.876649 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ccf118-3bfa-45d3-b34e-2584cde3a0c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:59.878705 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ccf118-3bfa-45d3-b34e-2584cde3a0c3 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:59.889820 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 906/1816] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:16:59 2022] PUT /identity/v3/projects/e3abc7d1ffc140cea2fd3e4afc677285/users/7acbd98fed99463ba2f672c7619d416b/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:59.900050 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 911/1817] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:16:59 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 16:16:59.901992 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7682c5e2-010c-4994-8c54-f7dc4c51d551 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:59.902281 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7682c5e2-010c-4994-8c54-f7dc4c51d551 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:59.902554 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7682c5e2-010c-4994-8c54-f7dc4c51d551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:59.905010 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ccf118-3bfa-45d3-b34e-2584cde3a0c3 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:59.927002 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ccf118-3bfa-45d3-b34e-2584cde3a0c3 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:59.952547 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-7682c5e2-010c-4994-8c54-f7dc4c51d551 None None] MFA Rules not processed for user `7acbd98fed99463ba2f672c7619d416b`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:59.953825 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-7682c5e2-010c-4994-8c54-f7dc4c51d551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:00.009550 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ffe6a9-3fd0-43d9-a1aa-35eea6725815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:00.010266 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ffe6a9-3fd0-43d9-a1aa-35eea6725815 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:00.010554 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ffe6a9-3fd0-43d9-a1aa-35eea6725815 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:00.010828 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ffe6a9-3fd0-43d9-a1aa-35eea6725815 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:00.011717 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ffe6a9-3fd0-43d9-a1aa-35eea6725815 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:00.013579 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ffe6a9-3fd0-43d9-a1aa-35eea6725815 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:00.035048 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 912/1818] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:16:59 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 16:17:00.041557 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ffe6a9-3fd0-43d9-a1aa-35eea6725815 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:00.052878 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 907/1819] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:16:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:17:00.059030 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7682c5e2-010c-4994-8c54-f7dc4c51d551 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:00.061869 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ffe6a9-3fd0-43d9-a1aa-35eea6725815 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:00.068605 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1858b4b7-c5ac-4864-b34a-6ff8e14c7987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:00.070342 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1858b4b7-c5ac-4864-b34a-6ff8e14c7987 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:00.070794 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1858b4b7-c5ac-4864-b34a-6ff8e14c7987 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:00.071064 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1858b4b7-c5ac-4864-b34a-6ff8e14c7987 admin admin] PATH_INFO: `/v3/projects/2472565b2a8944028c84aa0a8c06b79c/users/cba51e6aa03c4302b9a35579346d9159/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:00.073047 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7682c5e2-010c-4994-8c54-f7dc4c51d551 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:00.082014 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f96a04b-442f-4fd4-af7e-84b0f76a57e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:00.082842 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f96a04b-442f-4fd4-af7e-84b0f76a57e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:00.083211 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f96a04b-442f-4fd4-af7e-84b0f76a57e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:00.083654 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f96a04b-442f-4fd4-af7e-84b0f76a57e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:00.084370 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f96a04b-442f-4fd4-af7e-84b0f76a57e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:00.086539 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f96a04b-442f-4fd4-af7e-84b0f76a57e3 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:00.093251 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1858b4b7-c5ac-4864-b34a-6ff8e14c7987 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2472565b2a8944028c84aa0a8c06b79c, user_id=cba51e6aa03c4302b9a35579346d9159, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:00.097842 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1858b4b7-c5ac-4864-b34a-6ff8e14c7987 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:00.120862 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 913/1820] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:17:00 2022] PUT /identity/v3/projects/2472565b2a8944028c84aa0a8c06b79c/users/cba51e6aa03c4302b9a35579346d9159/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:00.130505 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc633c5b-1d32-472f-9cae-d7e318ea1006 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:00.130802 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc633c5b-1d32-472f-9cae-d7e318ea1006 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:00.131107 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc633c5b-1d32-472f-9cae-d7e318ea1006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:00.156668 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 908/1821] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:17:00 2022] POST /identity/v3/projects => generated 383 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:17:00.161571 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f96a04b-442f-4fd4-af7e-84b0f76a57e3 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:00.177024 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-cc633c5b-1d32-472f-9cae-d7e318ea1006 None None] MFA Rules not processed for user `cba51e6aa03c4302b9a35579346d9159`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:00.178454 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-cc633c5b-1d32-472f-9cae-d7e318ea1006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:00.250085 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:00.286681 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:00.293135 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f96a04b-442f-4fd4-af7e-84b0f76a57e3 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:00.294661 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 914/1822] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:17:00 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:00.302357 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc633c5b-1d32-472f-9cae-d7e318ea1006 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:00.316892 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830cc1a5-01bd-4aeb-8314-f8de411c731a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:00.318081 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830cc1a5-01bd-4aeb-8314-f8de411c731a admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:00.319558 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830cc1a5-01bd-4aeb-8314-f8de411c731a admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:00.320014 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830cc1a5-01bd-4aeb-8314-f8de411c731a admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:00.321078 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830cc1a5-01bd-4aeb-8314-f8de411c731a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:00.326206 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830cc1a5-01bd-4aeb-8314-f8de411c731a admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:00.328019 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc633c5b-1d32-472f-9cae-d7e318ea1006 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:00.339877 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6a99b0-2a24-4e88-a2e5-790f4f9201c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:00.343549 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6a99b0-2a24-4e88-a2e5-790f4f9201c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:00.344288 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6a99b0-2a24-4e88-a2e5-790f4f9201c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:00.344934 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6a99b0-2a24-4e88-a2e5-790f4f9201c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:00.347506 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6a99b0-2a24-4e88-a2e5-790f4f9201c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:00.356267 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6a99b0-2a24-4e88-a2e5-790f4f9201c9 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:00.366356 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 909/1823] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:17:00 2022] POST /identity/v3/users => generated 298 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:17:00.373315 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830cc1a5-01bd-4aeb-8314-f8de411c731a admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:00.392297 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830cc1a5-01bd-4aeb-8314-f8de411c731a admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:00.398183 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52420d75-7a91-41c1-9a4d-86bef220a391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:00.398987 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52420d75-7a91-41c1-9a4d-86bef220a391 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:00.399359 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52420d75-7a91-41c1-9a4d-86bef220a391 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:00.399634 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52420d75-7a91-41c1-9a4d-86bef220a391 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:00.400371 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52420d75-7a91-41c1-9a4d-86bef220a391 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:00.402141 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52420d75-7a91-41c1-9a4d-86bef220a391 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:00.417039 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 915/1824] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:17:00 2022] POST /identity/v3/projects => generated 387 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:17:00.421463 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6a99b0-2a24-4e88-a2e5-790f4f9201c9 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:00.429355 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 910/1825] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:17:00 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:17:00.434706 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52420d75-7a91-41c1-9a4d-86bef220a391 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:00.452405 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52420d75-7a91-41c1-9a4d-86bef220a391 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:00.526682 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6a99b0-2a24-4e88-a2e5-790f4f9201c9 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:00.535991 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd5e087-e54e-479b-b6eb-81bd35d1336d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:00.536831 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd5e087-e54e-479b-b6eb-81bd35d1336d admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:00.537220 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd5e087-e54e-479b-b6eb-81bd35d1336d admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:00.537571 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd5e087-e54e-479b-b6eb-81bd35d1336d admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:00.538308 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd5e087-e54e-479b-b6eb-81bd35d1336d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:00.541244 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd5e087-e54e-479b-b6eb-81bd35d1336d admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:00.552503 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66732e26-409b-4b13-a801-cb0d6e05bd3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:00.553616 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66732e26-409b-4b13-a801-cb0d6e05bd3e admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:00.554067 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66732e26-409b-4b13-a801-cb0d6e05bd3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:00.554543 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66732e26-409b-4b13-a801-cb0d6e05bd3e admin admin] PATH_INFO: `/v3/projects/d9f207d474314ca4a8c8083323c3a7ca/users/8e193a34ba9043baba0091c70e03659f/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:00.592934 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66732e26-409b-4b13-a801-cb0d6e05bd3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9f207d474314ca4a8c8083323c3a7ca, user_id=8e193a34ba9043baba0091c70e03659f, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:00.598852 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 916/1826] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:17:00 2022] POST /identity/v3/users => generated 300 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:17:00.599115 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66732e26-409b-4b13-a801-cb0d6e05bd3e admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:00.600368 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd5e087-e54e-479b-b6eb-81bd35d1336d admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:00.621844 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 911/1827] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:17:00 2022] PUT /identity/v3/projects/d9f207d474314ca4a8c8083323c3a7ca/users/8e193a34ba9043baba0091c70e03659f/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:17:00.626007 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66732e26-409b-4b13-a801-cb0d6e05bd3e admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:00.649739 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd5e087-e54e-479b-b6eb-81bd35d1336d admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:00.761151 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66732e26-409b-4b13-a801-cb0d6e05bd3e admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:00.772449 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5892e0fd-1ae9-4a3d-9bc1-78170141f7ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:00.773426 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5892e0fd-1ae9-4a3d-9bc1-78170141f7ac admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:00.773982 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5892e0fd-1ae9-4a3d-9bc1-78170141f7ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:00.774237 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5892e0fd-1ae9-4a3d-9bc1-78170141f7ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:00.775292 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5892e0fd-1ae9-4a3d-9bc1-78170141f7ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:00.777245 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5892e0fd-1ae9-4a3d-9bc1-78170141f7ac admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:00.793658 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 912/1828] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:17:00 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:17:00.797938 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5892e0fd-1ae9-4a3d-9bc1-78170141f7ac admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:00.814522 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5892e0fd-1ae9-4a3d-9bc1-78170141f7ac admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:00.821894 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b127e39-7eb8-4938-8fae-9ecd5e016c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:00.822620 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b127e39-7eb8-4938-8fae-9ecd5e016c87 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:00.822899 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b127e39-7eb8-4938-8fae-9ecd5e016c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:00.823224 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b127e39-7eb8-4938-8fae-9ecd5e016c87 admin admin] PATH_INFO: `/v3/projects/d9f207d474314ca4a8c8083323c3a7ca/users/8e193a34ba9043baba0091c70e03659f/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:00.826231 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b127e39-7eb8-4938-8fae-9ecd5e016c87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9f207d474314ca4a8c8083323c3a7ca, user_id=8e193a34ba9043baba0091c70e03659f, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:00.830065 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b127e39-7eb8-4938-8fae-9ecd5e016c87 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:00.845110 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 913/1829] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:17:00 2022] PUT /identity/v3/projects/d9f207d474314ca4a8c8083323c3a7ca/users/8e193a34ba9043baba0091c70e03659f/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:00.855973 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53769acd-11d1-4d9e-b42c-bbcb0254f9a2 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:00.856559 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53769acd-11d1-4d9e-b42c-bbcb0254f9a2 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:00.857062 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53769acd-11d1-4d9e-b42c-bbcb0254f9a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:00.862351 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e44398-d8c2-4cc3-8c25-b1c57993edff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:00.862998 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e44398-d8c2-4cc3-8c25-b1c57993edff admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:00.863276 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e44398-d8c2-4cc3-8c25-b1c57993edff admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:00.863548 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e44398-d8c2-4cc3-8c25-b1c57993edff admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:00.864222 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e44398-d8c2-4cc3-8c25-b1c57993edff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:00.865992 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e44398-d8c2-4cc3-8c25-b1c57993edff admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:00.892129 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 917/1830] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:17:00 2022] GET /identity/v3/roles => generated 1420 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:17:00.897571 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e44398-d8c2-4cc3-8c25-b1c57993edff admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:00.901252 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-53769acd-11d1-4d9e-b42c-bbcb0254f9a2 None None] MFA Rules not processed for user `8e193a34ba9043baba0091c70e03659f`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:00.902433 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-53769acd-11d1-4d9e-b42c-bbcb0254f9a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:00.926117 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e44398-d8c2-4cc3-8c25-b1c57993edff admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:00.946339 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:01.013117 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 914/1831] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:17:00 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:17:01.026872 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53769acd-11d1-4d9e-b42c-bbcb0254f9a2 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:01.028841 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-53769acd-11d1-4d9e-b42c-bbcb0254f9a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:01.052221 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:01.064694 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c60f48f-9636-4cd2-8fcd-d6ddec14d9d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:01.065920 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c60f48f-9636-4cd2-8fcd-d6ddec14d9d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:01.066385 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c60f48f-9636-4cd2-8fcd-d6ddec14d9d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:01.066799 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c60f48f-9636-4cd2-8fcd-d6ddec14d9d8 admin admin] PATH_INFO: `/v3/projects/aa9c370ad26944dd804db1e4f6473d45/users/5a12ecff87db4064920d14dee150c290/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:01.117680 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c60f48f-9636-4cd2-8fcd-d6ddec14d9d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa9c370ad26944dd804db1e4f6473d45, user_id=5a12ecff87db4064920d14dee150c290, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:01.121832 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c60f48f-9636-4cd2-8fcd-d6ddec14d9d8 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:01.133434 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53769acd-11d1-4d9e-b42c-bbcb0254f9a2 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:01.140179 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc0c914-3f5c-49e2-b725-d3595a659c56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:01.140747 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc0c914-3f5c-49e2-b725-d3595a659c56 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:01.141110 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc0c914-3f5c-49e2-b725-d3595a659c56 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:01.141408 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc0c914-3f5c-49e2-b725-d3595a659c56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:01.153276 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 918/1832] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:17:00 2022] PUT /identity/v3/projects/aa9c370ad26944dd804db1e4f6473d45/users/5a12ecff87db4064920d14dee150c290/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:17:01.157265 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c60f48f-9636-4cd2-8fcd-d6ddec14d9d8 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:01.256483 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc0c914-3f5c-49e2-b725-d3595a659c56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:01.258974 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc0c914-3f5c-49e2-b725-d3595a659c56 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:01.269482 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c60f48f-9636-4cd2-8fcd-d6ddec14d9d8 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:01.270134 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 915/1833] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:17:01 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:17:01.278816 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad89975-fbc6-4a20-8379-945c7def0d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:01.279778 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad89975-fbc6-4a20-8379-945c7def0d35 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:01.280253 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad89975-fbc6-4a20-8379-945c7def0d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:01.280672 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad89975-fbc6-4a20-8379-945c7def0d35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:01.281740 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad89975-fbc6-4a20-8379-945c7def0d35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:01.283507 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad89975-fbc6-4a20-8379-945c7def0d35 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:01.300491 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 919/1834] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:17:01 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:17:01.301561 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc0c914-3f5c-49e2-b725-d3595a659c56 service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:01.314107 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc0c914-3f5c-49e2-b725-d3595a659c56 service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:01.318906 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298ac0c9-8767-4587-a64b-74df5679f5fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:01.319655 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298ac0c9-8767-4587-a64b-74df5679f5fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:01.320005 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298ac0c9-8767-4587-a64b-74df5679f5fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:01.320345 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298ac0c9-8767-4587-a64b-74df5679f5fd admin admin] PATH_INFO: `/v3/projects/aa9c370ad26944dd804db1e4f6473d45/users/5a12ecff87db4064920d14dee150c290/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:01.322379 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298ac0c9-8767-4587-a64b-74df5679f5fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa9c370ad26944dd804db1e4f6473d45, user_id=5a12ecff87db4064920d14dee150c290, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:01.325269 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298ac0c9-8767-4587-a64b-74df5679f5fd admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:01.341806 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 916/1835] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:17:01 2022] PUT /identity/v3/projects/aa9c370ad26944dd804db1e4f6473d45/users/5a12ecff87db4064920d14dee150c290/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:17:01.355441 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bba5ff-e19e-47c0-96eb-fda2b4de20ae None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:01.355944 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bba5ff-e19e-47c0-96eb-fda2b4de20ae None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:01.356595 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bba5ff-e19e-47c0-96eb-fda2b4de20ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:01.403110 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-72bba5ff-e19e-47c0-96eb-fda2b4de20ae None None] MFA Rules not processed for user `5a12ecff87db4064920d14dee150c290`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:01.404574 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-72bba5ff-e19e-47c0-96eb-fda2b4de20ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:01.525064 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 920/1836] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:17:01 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:01.714002 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298ac0c9-8767-4587-a64b-74df5679f5fd admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:01.715979 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-298ac0c9-8767-4587-a64b-74df5679f5fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:01.803603 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298ac0c9-8767-4587-a64b-74df5679f5fd admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:01.808610 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068f9425-e702-4ffd-9f6b-550059ddac00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:01.809393 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068f9425-e702-4ffd-9f6b-550059ddac00 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:01.809769 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068f9425-e702-4ffd-9f6b-550059ddac00 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:01.810143 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068f9425-e702-4ffd-9f6b-550059ddac00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:01.825206 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068f9425-e702-4ffd-9f6b-550059ddac00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:01.827580 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068f9425-e702-4ffd-9f6b-550059ddac00 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:01.850763 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 917/1837] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:17:01 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:17:01.980183 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bba5ff-e19e-47c0-96eb-fda2b4de20ae None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:02.064243 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:02.074290 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bba5ff-e19e-47c0-96eb-fda2b4de20ae None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:02.088827 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcc1de1-071f-45dc-b179-b17b12a454bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:02.089567 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcc1de1-071f-45dc-b179-b17b12a454bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:02.089748 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcc1de1-071f-45dc-b179-b17b12a454bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:02.090190 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcc1de1-071f-45dc-b179-b17b12a454bc admin admin] PATH_INFO: `/v3/users/cba51e6aa03c4302b9a35579346d9159` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:02.091548 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcc1de1-071f-45dc-b179-b17b12a454bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=cba51e6aa03c4302b9a35579346d9159)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:02.093675 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcc1de1-071f-45dc-b179-b17b12a454bc admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:02.184732 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-dbcc1de1-071f-45dc-b179-b17b12a454bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cba51e6aa03c4302b9a35579346d9159'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:02.185705 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-dbcc1de1-071f-45dc-b179-b17b12a454bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cba51e6aa03c4302b9a35579346d9159'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:02.197880 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-dbcc1de1-071f-45dc-b179-b17b12a454bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cba51e6aa03c4302b9a35579346d9159'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:02.211739 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-dbcc1de1-071f-45dc-b179-b17b12a454bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cba51e6aa03c4302b9a35579346d9159'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:02.219319 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-dbcc1de1-071f-45dc-b179-b17b12a454bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cba51e6aa03c4302b9a35579346d9159'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:02.230105 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 921/1838] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:17:01 2022] DELETE /identity/v3/users/cba51e6aa03c4302b9a35579346d9159 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:02.232210 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d5672-42e6-41ca-bad8-6cfcc2f4ddfe None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:02.232679 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d5672-42e6-41ca-bad8-6cfcc2f4ddfe None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:02.232868 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d5672-42e6-41ca-bad8-6cfcc2f4ddfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:02.257647 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-8b9d5672-42e6-41ca-bad8-6cfcc2f4ddfe None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:02.258800 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-8b9d5672-42e6-41ca-bad8-6cfcc2f4ddfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:02.267964 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:02.375193 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 918/1839] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:17:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:02.386963 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcc1de1-071f-45dc-b179-b17b12a454bc admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:02.388389 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-dbcc1de1-071f-45dc-b179-b17b12a454bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:02.460865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcc1de1-071f-45dc-b179-b17b12a454bc admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:02.467133 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f2454c-aee1-4b69-91ac-8eb5929e5358 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:02.468043 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f2454c-aee1-4b69-91ac-8eb5929e5358 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:02.468328 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f2454c-aee1-4b69-91ac-8eb5929e5358 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:02.468657 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f2454c-aee1-4b69-91ac-8eb5929e5358 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:02.480699 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f2454c-aee1-4b69-91ac-8eb5929e5358 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:02.482551 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f2454c-aee1-4b69-91ac-8eb5929e5358 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:02.494243 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 922/1840] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:17:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:17:02.839741 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9d5672-42e6-41ca-bad8-6cfcc2f4ddfe None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:02.841577 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-8b9d5672-42e6-41ca-bad8-6cfcc2f4ddfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:02.859097 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9d5672-42e6-41ca-bad8-6cfcc2f4ddfe None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:02.866697 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e370775-8852-4847-81d1-215059843b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:02.882818 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e370775-8852-4847-81d1-215059843b06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:02.883111 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e370775-8852-4847-81d1-215059843b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:02.883395 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e370775-8852-4847-81d1-215059843b06 admin admin] PATH_INFO: `/v3/projects/2472565b2a8944028c84aa0a8c06b79c` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:02.884794 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e370775-8852-4847-81d1-215059843b06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2472565b2a8944028c84aa0a8c06b79c)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:02.886757 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e370775-8852-4847-81d1-215059843b06 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:02.945149 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8e370775-8852-4847-81d1-215059843b06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2472565b2a8944028c84aa0a8c06b79c', 'request_id': 'req-8e370775-8852-4847-81d1-215059843b06'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:02.951538 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8e370775-8852-4847-81d1-215059843b06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2472565b2a8944028c84aa0a8c06b79c', 'request_id': 'req-8e370775-8852-4847-81d1-215059843b06'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:02.961459 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8e370775-8852-4847-81d1-215059843b06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2472565b2a8944028c84aa0a8c06b79c', 'request_id': 'req-8e370775-8852-4847-81d1-215059843b06'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:02.974016 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8e370775-8852-4847-81d1-215059843b06 admin admin] The token cache is being invalidate because project 2472565b2a8944028c84aa0a8c06b79c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:17:02.974317 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8e370775-8852-4847-81d1-215059843b06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:02.974976 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8e370775-8852-4847-81d1-215059843b06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:02.976493 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 919/1841] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:17:02 2022] DELETE /identity/v3/projects/2472565b2a8944028c84aa0a8c06b79c => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:02.980639 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f2454c-aee1-4b69-91ac-8eb5929e5358 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:02.990987 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f2454c-aee1-4b69-91ac-8eb5929e5358 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:03.057325 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aa37dc-9aa2-47b1-8ef8-b1172cf63240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:03.057961 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa37dc-9aa2-47b1-8ef8-b1172cf63240 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:03.058257 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa37dc-9aa2-47b1-8ef8-b1172cf63240 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:03.058507 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa37dc-9aa2-47b1-8ef8-b1172cf63240 admin admin] PATH_INFO: `/v3/users/5a12ecff87db4064920d14dee150c290` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:03.059752 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aa37dc-9aa2-47b1-8ef8-b1172cf63240 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a12ecff87db4064920d14dee150c290)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:03.061682 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aa37dc-9aa2-47b1-8ef8-b1172cf63240 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:03.087607 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:03.118789 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-f0aa37dc-9aa2-47b1-8ef8-b1172cf63240 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a12ecff87db4064920d14dee150c290'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:03.119458 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-f0aa37dc-9aa2-47b1-8ef8-b1172cf63240 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a12ecff87db4064920d14dee150c290'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:03.126123 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-f0aa37dc-9aa2-47b1-8ef8-b1172cf63240 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a12ecff87db4064920d14dee150c290'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:03.142448 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-f0aa37dc-9aa2-47b1-8ef8-b1172cf63240 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a12ecff87db4064920d14dee150c290'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:03.154458 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-f0aa37dc-9aa2-47b1-8ef8-b1172cf63240 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a12ecff87db4064920d14dee150c290'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:03.161501 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 923/1842] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:17:02 2022] DELETE /identity/v3/users/5a12ecff87db4064920d14dee150c290 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:03.648769 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e370775-8852-4847-81d1-215059843b06 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:03.650277 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-8e370775-8852-4847-81d1-215059843b06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:03.665929 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e370775-8852-4847-81d1-215059843b06 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:03.733984 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8186ad90-b127-42d0-84a4-0304c28740b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:03.734925 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8186ad90-b127-42d0-84a4-0304c28740b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:03.735283 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8186ad90-b127-42d0-84a4-0304c28740b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:03.735614 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8186ad90-b127-42d0-84a4-0304c28740b4 admin admin] PATH_INFO: `/v3/projects/aa9c370ad26944dd804db1e4f6473d45` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:03.737011 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8186ad90-b127-42d0-84a4-0304c28740b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa9c370ad26944dd804db1e4f6473d45)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:03.738941 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8186ad90-b127-42d0-84a4-0304c28740b4 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:03.783443 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:03.796002 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8186ad90-b127-42d0-84a4-0304c28740b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa9c370ad26944dd804db1e4f6473d45', 'request_id': 'req-8186ad90-b127-42d0-84a4-0304c28740b4'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:03.802633 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8186ad90-b127-42d0-84a4-0304c28740b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa9c370ad26944dd804db1e4f6473d45', 'request_id': 'req-8186ad90-b127-42d0-84a4-0304c28740b4'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:03.814466 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8186ad90-b127-42d0-84a4-0304c28740b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa9c370ad26944dd804db1e4f6473d45', 'request_id': 'req-8186ad90-b127-42d0-84a4-0304c28740b4'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:03.828426 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8186ad90-b127-42d0-84a4-0304c28740b4 admin admin] The token cache is being invalidate because project aa9c370ad26944dd804db1e4f6473d45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:17:03.828711 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8186ad90-b127-42d0-84a4-0304c28740b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:03.829551 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8186ad90-b127-42d0-84a4-0304c28740b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:03.831209 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 920/1843] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:17:03 2022] DELETE /identity/v3/projects/aa9c370ad26944dd804db1e4f6473d45 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:03.836197 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbff0d9-29c4-4613-b078-7dfae572bb29 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:03.836505 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbff0d9-29c4-4613-b078-7dfae572bb29 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:03.836813 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbff0d9-29c4-4613-b078-7dfae572bb29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:03.858865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-ccbff0d9-29c4-4613-b078-7dfae572bb29 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:03.860081 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ccbff0d9-29c4-4613-b078-7dfae572bb29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:03.934224 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 924/1844] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:17:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:03.942511 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2382d3-92bb-4484-a90b-16be48e09eb3 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:03.942925 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2382d3-92bb-4484-a90b-16be48e09eb3 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:03.943151 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2382d3-92bb-4484-a90b-16be48e09eb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:03.968938 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-9f2382d3-92bb-4484-a90b-16be48e09eb3 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:03.970392 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-9f2382d3-92bb-4484-a90b-16be48e09eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:03.980205 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 921/1845] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:17:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:03.984449 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbff0d9-29c4-4613-b078-7dfae572bb29 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:03.994841 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbff0d9-29c4-4613-b078-7dfae572bb29 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:04.000894 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d250c09-7eff-4d37-9c84-022def5bc563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:04.001670 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d250c09-7eff-4d37-9c84-022def5bc563 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:04.002191 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d250c09-7eff-4d37-9c84-022def5bc563 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:04.003074 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d250c09-7eff-4d37-9c84-022def5bc563 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:04.006516 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d250c09-7eff-4d37-9c84-022def5bc563 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:17:04.006972 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d250c09-7eff-4d37-9c84-022def5bc563 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:04.009665 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d250c09-7eff-4d37-9c84-022def5bc563 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:04.024662 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 925/1846] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 6 16:17:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:17:04.031888 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2382d3-92bb-4484-a90b-16be48e09eb3 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:04.047380 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2382d3-92bb-4484-a90b-16be48e09eb3 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:04.054326 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8422bd-ed96-4aa2-b04b-5da5b542983b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:04.055039 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8422bd-ed96-4aa2-b04b-5da5b542983b admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:04.055385 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8422bd-ed96-4aa2-b04b-5da5b542983b admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:04.055538 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8422bd-ed96-4aa2-b04b-5da5b542983b admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:04.056303 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8422bd-ed96-4aa2-b04b-5da5b542983b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:04.058400 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8422bd-ed96-4aa2-b04b-5da5b542983b admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:04.105044 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 922/1847] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:17:04 2022] POST /identity/v3/projects => generated 385 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:17:04.112124 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d250c09-7eff-4d37-9c84-022def5bc563 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:04.124784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d250c09-7eff-4d37-9c84-022def5bc563 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:04.208090 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33322f57-befb-4d0f-a65c-7923aa9817fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:04.209087 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33322f57-befb-4d0f-a65c-7923aa9817fb admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:04.209423 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33322f57-befb-4d0f-a65c-7923aa9817fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:04.209812 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33322f57-befb-4d0f-a65c-7923aa9817fb admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:04.210514 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33322f57-befb-4d0f-a65c-7923aa9817fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:04.212635 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33322f57-befb-4d0f-a65c-7923aa9817fb admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:04.249352 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 926/1848] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:17:04 2022] POST /identity/v3/users => generated 293 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:17:04.250176 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8422bd-ed96-4aa2-b04b-5da5b542983b admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:04.261856 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8422bd-ed96-4aa2-b04b-5da5b542983b admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:04.268089 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c4ee1e-22d0-4aa6-8f6b-21b9e3a7bf33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:04.268794 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4ee1e-22d0-4aa6-8f6b-21b9e3a7bf33 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:04.269306 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4ee1e-22d0-4aa6-8f6b-21b9e3a7bf33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:04.269763 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4ee1e-22d0-4aa6-8f6b-21b9e3a7bf33 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:04.270500 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c4ee1e-22d0-4aa6-8f6b-21b9e3a7bf33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:04.272334 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c4ee1e-22d0-4aa6-8f6b-21b9e3a7bf33 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:04.285667 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 923/1849] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:17:04 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:17:04.290472 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33322f57-befb-4d0f-a65c-7923aa9817fb admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:04.301271 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33322f57-befb-4d0f-a65c-7923aa9817fb admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:04.310702 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cea8d4e-55fc-4a3f-a339-7535b21b5acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:04.311759 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cea8d4e-55fc-4a3f-a339-7535b21b5acd admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:04.312809 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cea8d4e-55fc-4a3f-a339-7535b21b5acd admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:04.313124 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cea8d4e-55fc-4a3f-a339-7535b21b5acd admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:04.313860 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cea8d4e-55fc-4a3f-a339-7535b21b5acd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:04.315822 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cea8d4e-55fc-4a3f-a339-7535b21b5acd admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:04.333559 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 927/1850] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:17:04 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 16:17:04.335221 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c4ee1e-22d0-4aa6-8f6b-21b9e3a7bf33 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:04.354787 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c4ee1e-22d0-4aa6-8f6b-21b9e3a7bf33 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:04.362825 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0368d6-16fe-4306-9e09-3734094764a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:04.363935 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0368d6-16fe-4306-9e09-3734094764a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:04.364555 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0368d6-16fe-4306-9e09-3734094764a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:04.365119 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0368d6-16fe-4306-9e09-3734094764a4 admin admin] PATH_INFO: `/v3/projects/7dcd024ced6a46bd9541eb3de7ce5a20/users/c1a01b383dce471c87424cde937d74c8/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:04.402057 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0368d6-16fe-4306-9e09-3734094764a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7dcd024ced6a46bd9541eb3de7ce5a20, user_id=c1a01b383dce471c87424cde937d74c8, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:04.406751 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0368d6-16fe-4306-9e09-3734094764a4 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:04.439434 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 924/1851] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:17:04 2022] PUT /identity/v3/projects/7dcd024ced6a46bd9541eb3de7ce5a20/users/c1a01b383dce471c87424cde937d74c8/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:04.442420 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3799fc3-af5b-465c-8ce3-40a66e6c0f27 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:04.442862 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3799fc3-af5b-465c-8ce3-40a66e6c0f27 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:04.443357 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3799fc3-af5b-465c-8ce3-40a66e6c0f27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:04.464609 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:04.528572 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-f3799fc3-af5b-465c-8ce3-40a66e6c0f27 None None] MFA Rules not processed for user `c1a01b383dce471c87424cde937d74c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:04.529938 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f3799fc3-af5b-465c-8ce3-40a66e6c0f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:04.618855 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 928/1852] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:17:04 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:04.634663 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0368d6-16fe-4306-9e09-3734094764a4 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:04.636427 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-2b0368d6-16fe-4306-9e09-3734094764a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:04.731072 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0368d6-16fe-4306-9e09-3734094764a4 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:04.738209 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549b6e80-8f43-42c1-b4ab-41ec448542d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:04.738209 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549b6e80-8f43-42c1-b4ab-41ec448542d7 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:04.738209 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549b6e80-8f43-42c1-b4ab-41ec448542d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:04.738209 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549b6e80-8f43-42c1-b4ab-41ec448542d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:04.750605 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549b6e80-8f43-42c1-b4ab-41ec448542d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:04.752562 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549b6e80-8f43-42c1-b4ab-41ec448542d7 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:04.765183 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 925/1853] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:17:04 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:17:12.561865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3799fc3-af5b-465c-8ce3-40a66e6c0f27 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:12.564007 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f3799fc3-af5b-465c-8ce3-40a66e6c0f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:12.577365 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3799fc3-af5b-465c-8ce3-40a66e6c0f27 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:12.581433 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2654f562-4c38-46e9-b548-25056d014555 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:12.583842 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2654f562-4c38-46e9-b548-25056d014555 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:12.584433 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2654f562-4c38-46e9-b548-25056d014555 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:12.584712 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2654f562-4c38-46e9-b548-25056d014555 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:12.586590 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-2654f562-4c38-46e9-b548-25056d014555 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:12.660989 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2654f562-4c38-46e9-b548-25056d014555 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:12.662599 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2654f562-4c38-46e9-b548-25056d014555 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:12.674774 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 929/1854] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:17:12 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:17:14.759896 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=64,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:19.480088 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549b6e80-8f43-42c1-b4ab-41ec448542d7 service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:19.481710 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-549b6e80-8f43-42c1-b4ab-41ec448542d7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:19.552281 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549b6e80-8f43-42c1-b4ab-41ec448542d7 service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:19.557720 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3263babb-b56e-4ce9-b385-7fc6a5fd8686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:19.558367 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3263babb-b56e-4ce9-b385-7fc6a5fd8686 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:19.558650 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3263babb-b56e-4ce9-b385-7fc6a5fd8686 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:19.558919 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3263babb-b56e-4ce9-b385-7fc6a5fd8686 admin admin] PATH_INFO: `/v3/users/9171d03379714b13a4b3fdd64a54f56a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:19.560547 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3263babb-b56e-4ce9-b385-7fc6a5fd8686 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9171d03379714b13a4b3fdd64a54f56a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:19.562760 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3263babb-b56e-4ce9-b385-7fc6a5fd8686 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:19.617601 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-3263babb-b56e-4ce9-b385-7fc6a5fd8686 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9171d03379714b13a4b3fdd64a54f56a'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:19.618208 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-3263babb-b56e-4ce9-b385-7fc6a5fd8686 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9171d03379714b13a4b3fdd64a54f56a'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:19.628429 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-3263babb-b56e-4ce9-b385-7fc6a5fd8686 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9171d03379714b13a4b3fdd64a54f56a'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:19.645704 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-3263babb-b56e-4ce9-b385-7fc6a5fd8686 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9171d03379714b13a4b3fdd64a54f56a'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:19.646511 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-3263babb-b56e-4ce9-b385-7fc6a5fd8686 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9171d03379714b13a4b3fdd64a54f56a'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:19.663228 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 926/1855] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:17:19 2022] DELETE /identity/v3/users/9171d03379714b13a4b3fdd64a54f56a => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:19.668126 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd33e13-5a41-469c-bc6d-073cbb24c9d3 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:19.668693 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd33e13-5a41-469c-bc6d-073cbb24c9d3 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:19.668904 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd33e13-5a41-469c-bc6d-073cbb24c9d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:19.691667 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-8cd33e13-5a41-469c-bc6d-073cbb24c9d3 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:19.693124 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8cd33e13-5a41-469c-bc6d-073cbb24c9d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:19.812982 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 930/1856] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:17:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:19.825032 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3263babb-b56e-4ce9-b385-7fc6a5fd8686 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:19.826552 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3263babb-b56e-4ce9-b385-7fc6a5fd8686 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:19.894973 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3263babb-b56e-4ce9-b385-7fc6a5fd8686 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:19.899752 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5999b5e-901f-498a-ae54-1b06c3efe29a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:19.900794 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5999b5e-901f-498a-ae54-1b06c3efe29a service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:19.900794 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5999b5e-901f-498a-ae54-1b06c3efe29a service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:19.901215 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5999b5e-901f-498a-ae54-1b06c3efe29a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:19.915498 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5999b5e-901f-498a-ae54-1b06c3efe29a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:19.917008 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5999b5e-901f-498a-ae54-1b06c3efe29a service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:19.929633 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 927/1857] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:17:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:17:20.249309 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd33e13-5a41-469c-bc6d-073cbb24c9d3 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:20.251202 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8cd33e13-5a41-469c-bc6d-073cbb24c9d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:20.264892 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd33e13-5a41-469c-bc6d-073cbb24c9d3 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:20.269905 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c77665-d600-4708-ad9c-63e4ef38c10c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:20.270650 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c77665-d600-4708-ad9c-63e4ef38c10c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:20.270845 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c77665-d600-4708-ad9c-63e4ef38c10c admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:20.271112 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c77665-d600-4708-ad9c-63e4ef38c10c admin admin] PATH_INFO: `/v3/projects/29659e2d27344b079fa71e73e707726d` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:20.272280 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c77665-d600-4708-ad9c-63e4ef38c10c admin admin] RBAC: Authorizing `identity:delete_project(project_id=29659e2d27344b079fa71e73e707726d)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:20.274208 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c77665-d600-4708-ad9c-63e4ef38c10c admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:20.299240 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:20.322813 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-16c77665-d600-4708-ad9c-63e4ef38c10c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29659e2d27344b079fa71e73e707726d', 'request_id': 'req-16c77665-d600-4708-ad9c-63e4ef38c10c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:20.331203 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-16c77665-d600-4708-ad9c-63e4ef38c10c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29659e2d27344b079fa71e73e707726d', 'request_id': 'req-16c77665-d600-4708-ad9c-63e4ef38c10c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:20.336999 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-16c77665-d600-4708-ad9c-63e4ef38c10c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29659e2d27344b079fa71e73e707726d', 'request_id': 'req-16c77665-d600-4708-ad9c-63e4ef38c10c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:20.347960 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-16c77665-d600-4708-ad9c-63e4ef38c10c admin admin] The token cache is being invalidate because project 29659e2d27344b079fa71e73e707726d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:17:20.348246 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-16c77665-d600-4708-ad9c-63e4ef38c10c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:20.348997 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-16c77665-d600-4708-ad9c-63e4ef38c10c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:20.350559 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 931/1858] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:17:20 2022] DELETE /identity/v3/projects/29659e2d27344b079fa71e73e707726d => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:20.354753 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5999b5e-901f-498a-ae54-1b06c3efe29a service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:20.365219 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5999b5e-901f-498a-ae54-1b06c3efe29a service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:20.385929 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:20.430085 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e31717-79ac-4e38-9859-6fd02a2952d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:20.430727 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e31717-79ac-4e38-9859-6fd02a2952d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:20.431013 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e31717-79ac-4e38-9859-6fd02a2952d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:20.431281 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e31717-79ac-4e38-9859-6fd02a2952d8 admin admin] PATH_INFO: `/v3/users/34d1a67a7d3a4802abca489173c4ca95` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:20.432422 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e31717-79ac-4e38-9859-6fd02a2952d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34d1a67a7d3a4802abca489173c4ca95)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:20.434410 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e31717-79ac-4e38-9859-6fd02a2952d8 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:20.482430 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e3e31717-79ac-4e38-9859-6fd02a2952d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34d1a67a7d3a4802abca489173c4ca95'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:20.483499 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e3e31717-79ac-4e38-9859-6fd02a2952d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34d1a67a7d3a4802abca489173c4ca95'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:20.490506 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e3e31717-79ac-4e38-9859-6fd02a2952d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34d1a67a7d3a4802abca489173c4ca95'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:20.502842 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e3e31717-79ac-4e38-9859-6fd02a2952d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34d1a67a7d3a4802abca489173c4ca95'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:20.503878 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e3e31717-79ac-4e38-9859-6fd02a2952d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34d1a67a7d3a4802abca489173c4ca95'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:20.517989 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 928/1859] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:17:20 2022] DELETE /identity/v3/users/34d1a67a7d3a4802abca489173c4ca95 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:17:20.795560 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c77665-d600-4708-ad9c-63e4ef38c10c admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:20.797111 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-16c77665-d600-4708-ad9c-63e4ef38c10c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:20.816547 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c77665-d600-4708-ad9c-63e4ef38c10c admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:20.876390 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7369639-291d-48ed-923e-c6755c8464b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:20.877167 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7369639-291d-48ed-923e-c6755c8464b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:20.877336 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7369639-291d-48ed-923e-c6755c8464b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:20.877589 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7369639-291d-48ed-923e-c6755c8464b9 admin admin] PATH_INFO: `/v3/projects/8cce38d3f06f45dba538a8940c63efc6` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:20.878896 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7369639-291d-48ed-923e-c6755c8464b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cce38d3f06f45dba538a8940c63efc6)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:20.881645 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7369639-291d-48ed-923e-c6755c8464b9 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:20.929187 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e7369639-291d-48ed-923e-c6755c8464b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cce38d3f06f45dba538a8940c63efc6', 'request_id': 'req-e7369639-291d-48ed-923e-c6755c8464b9'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:20.938214 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e7369639-291d-48ed-923e-c6755c8464b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cce38d3f06f45dba538a8940c63efc6', 'request_id': 'req-e7369639-291d-48ed-923e-c6755c8464b9'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:20.946042 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e7369639-291d-48ed-923e-c6755c8464b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cce38d3f06f45dba538a8940c63efc6', 'request_id': 'req-e7369639-291d-48ed-923e-c6755c8464b9'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:20.959086 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e7369639-291d-48ed-923e-c6755c8464b9 admin admin] The token cache is being invalidate because project 8cce38d3f06f45dba538a8940c63efc6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:17:20.959387 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e7369639-291d-48ed-923e-c6755c8464b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:20.960098 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e7369639-291d-48ed-923e-c6755c8464b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:20.961719 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 932/1860] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:17:20 2022] DELETE /identity/v3/projects/8cce38d3f06f45dba538a8940c63efc6 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:20.966796 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a5006-1bb2-4ea7-aedc-d3ec4391cc80 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:20.967086 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a5006-1bb2-4ea7-aedc-d3ec4391cc80 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:20.967369 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a5006-1bb2-4ea7-aedc-d3ec4391cc80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:20.988161 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-9e4a5006-1bb2-4ea7-aedc-d3ec4391cc80 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:20.989432 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-9e4a5006-1bb2-4ea7-aedc-d3ec4391cc80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:21.051592 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 929/1861] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:17:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:21.059045 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f76c8e-f709-4abc-82a5-3e319b13faf1 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:21.059329 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f76c8e-f709-4abc-82a5-3e319b13faf1 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:21.059601 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f76c8e-f709-4abc-82a5-3e319b13faf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:21.080502 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-83f76c8e-f709-4abc-82a5-3e319b13faf1 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:21.081727 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-83f76c8e-f709-4abc-82a5-3e319b13faf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:21.089724 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 933/1862] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:17:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:21.093873 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4a5006-1bb2-4ea7-aedc-d3ec4391cc80 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:21.104130 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4a5006-1bb2-4ea7-aedc-d3ec4391cc80 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:21.109202 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c105ca70-af02-48a7-91ca-f6cf5abb56b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:21.109809 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c105ca70-af02-48a7-91ca-f6cf5abb56b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:21.110087 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c105ca70-af02-48a7-91ca-f6cf5abb56b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:21.110362 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c105ca70-af02-48a7-91ca-f6cf5abb56b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:21.111017 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c105ca70-af02-48a7-91ca-f6cf5abb56b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:17:21.111318 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c105ca70-af02-48a7-91ca-f6cf5abb56b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:21.113055 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c105ca70-af02-48a7-91ca-f6cf5abb56b7 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:21.124849 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 930/1863] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 6 16:17:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:17:21.129377 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f76c8e-f709-4abc-82a5-3e319b13faf1 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:21.138873 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f76c8e-f709-4abc-82a5-3e319b13faf1 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:21.144423 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68a4b9a-8e26-4ba8-bbd6-fde8f4400587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:21.145079 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68a4b9a-8e26-4ba8-bbd6-fde8f4400587 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:21.145357 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68a4b9a-8e26-4ba8-bbd6-fde8f4400587 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:21.145621 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68a4b9a-8e26-4ba8-bbd6-fde8f4400587 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:21.146268 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68a4b9a-8e26-4ba8-bbd6-fde8f4400587 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:21.148219 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68a4b9a-8e26-4ba8-bbd6-fde8f4400587 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:21.180278 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 934/1864] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:17:21 2022] POST /identity/v3/projects => generated 365 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:17:21.181372 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c105ca70-af02-48a7-91ca-f6cf5abb56b7 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:21.191882 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c105ca70-af02-48a7-91ca-f6cf5abb56b7 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:21.220136 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:21.253166 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6168abf8-9b07-419d-a72a-59d29af876a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:21.253836 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6168abf8-9b07-419d-a72a-59d29af876a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:21.254119 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6168abf8-9b07-419d-a72a-59d29af876a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:21.254404 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6168abf8-9b07-419d-a72a-59d29af876a6 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:21.259283 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6168abf8-9b07-419d-a72a-59d29af876a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:21.261411 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6168abf8-9b07-419d-a72a-59d29af876a6 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:21.287065 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 931/1865] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:17:21 2022] POST /identity/v3/users => generated 283 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:17:21.291298 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68a4b9a-8e26-4ba8-bbd6-fde8f4400587 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:21.303175 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68a4b9a-8e26-4ba8-bbd6-fde8f4400587 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:21.308973 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6f7546-ebd9-4b64-8ac9-5401f2c91786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:21.309679 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f7546-ebd9-4b64-8ac9-5401f2c91786 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:21.309961 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f7546-ebd9-4b64-8ac9-5401f2c91786 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:21.310232 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f7546-ebd9-4b64-8ac9-5401f2c91786 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:21.310895 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6f7546-ebd9-4b64-8ac9-5401f2c91786 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:21.312876 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6f7546-ebd9-4b64-8ac9-5401f2c91786 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:21.324172 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 935/1866] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:17:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:17:21.328491 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6168abf8-9b07-419d-a72a-59d29af876a6 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:21.339224 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6168abf8-9b07-419d-a72a-59d29af876a6 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:21.345646 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70626e54-7313-42d9-8da0-dad77add52dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:21.346319 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70626e54-7313-42d9-8da0-dad77add52dd admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:21.346549 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70626e54-7313-42d9-8da0-dad77add52dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:21.346843 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70626e54-7313-42d9-8da0-dad77add52dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:21.347453 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70626e54-7313-42d9-8da0-dad77add52dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:21.349184 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70626e54-7313-42d9-8da0-dad77add52dd admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:21.362088 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 932/1867] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:17:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:17:21.366894 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6f7546-ebd9-4b64-8ac9-5401f2c91786 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:21.372242 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:21.378859 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6f7546-ebd9-4b64-8ac9-5401f2c91786 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:21.385497 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d887649-e8ab-4aa6-88c2-59d3167af958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:21.386160 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d887649-e8ab-4aa6-88c2-59d3167af958 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:21.386439 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d887649-e8ab-4aa6-88c2-59d3167af958 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:21.386713 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d887649-e8ab-4aa6-88c2-59d3167af958 admin admin] PATH_INFO: `/v3/projects/8346a87933d34cbfae0857f49eb8e959/users/c8447ea02f4444b2bd165dbb30b3031b/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:21.402743 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d887649-e8ab-4aa6-88c2-59d3167af958 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8346a87933d34cbfae0857f49eb8e959, user_id=c8447ea02f4444b2bd165dbb30b3031b, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:21.405471 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d887649-e8ab-4aa6-88c2-59d3167af958 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:21.418987 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 936/1868] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:17:21 2022] PUT /identity/v3/projects/8346a87933d34cbfae0857f49eb8e959/users/c8447ea02f4444b2bd165dbb30b3031b/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:21.431289 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d697ef8e-e780-4d44-802c-f7e060e09f42 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:21.431499 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d697ef8e-e780-4d44-802c-f7e060e09f42 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:21.431829 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d697ef8e-e780-4d44-802c-f7e060e09f42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:21.462067 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-d697ef8e-e780-4d44-802c-f7e060e09f42 None None] MFA Rules not processed for user `c8447ea02f4444b2bd165dbb30b3031b`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:21.463319 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d697ef8e-e780-4d44-802c-f7e060e09f42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:21.529974 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 933/1869] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:17:21 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:21.535439 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d887649-e8ab-4aa6-88c2-59d3167af958 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:21.599362 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d887649-e8ab-4aa6-88c2-59d3167af958 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:21.604117 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1931ce67-5724-4138-908c-fe67f6b28140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:21.604738 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1931ce67-5724-4138-908c-fe67f6b28140 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:21.605047 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1931ce67-5724-4138-908c-fe67f6b28140 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:21.605443 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1931ce67-5724-4138-908c-fe67f6b28140 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:21.606150 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1931ce67-5724-4138-908c-fe67f6b28140 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:21.608091 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1931ce67-5724-4138-908c-fe67f6b28140 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:21.643394 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 937/1870] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:17:21 2022] POST /identity/v3/projects => generated 365 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:17:21.644607 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d697ef8e-e780-4d44-802c-f7e060e09f42 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:21.655838 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d697ef8e-e780-4d44-802c-f7e060e09f42 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:21.716805 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882c95b5-353b-49b6-9a9f-5d8b50918851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:21.717429 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882c95b5-353b-49b6-9a9f-5d8b50918851 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:21.717709 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882c95b5-353b-49b6-9a9f-5d8b50918851 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:21.717977 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882c95b5-353b-49b6-9a9f-5d8b50918851 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:21.718624 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882c95b5-353b-49b6-9a9f-5d8b50918851 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:21.720559 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882c95b5-353b-49b6-9a9f-5d8b50918851 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:21.742626 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 934/1871] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:17:21 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:17:21.747056 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1931ce67-5724-4138-908c-fe67f6b28140 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:21.757990 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1931ce67-5724-4138-908c-fe67f6b28140 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:21.763025 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf64fb8-cd43-462c-9976-b0f1deb61066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:21.763630 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf64fb8-cd43-462c-9976-b0f1deb61066 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:21.763908 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf64fb8-cd43-462c-9976-b0f1deb61066 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:21.764176 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf64fb8-cd43-462c-9976-b0f1deb61066 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:21.764884 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf64fb8-cd43-462c-9976-b0f1deb61066 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:21.766454 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf64fb8-cd43-462c-9976-b0f1deb61066 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:21.777850 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 938/1872] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:17:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:17:21.782939 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882c95b5-353b-49b6-9a9f-5d8b50918851 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:21.793517 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882c95b5-353b-49b6-9a9f-5d8b50918851 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:21.798781 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6965f613-0aca-456b-a504-7e51cd6ad655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:21.799433 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6965f613-0aca-456b-a504-7e51cd6ad655 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:21.799711 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6965f613-0aca-456b-a504-7e51cd6ad655 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:21.799977 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6965f613-0aca-456b-a504-7e51cd6ad655 admin admin] PATH_INFO: `/v3/projects/bc2eab963eae48158a0f623f14731142/users/2160debbd32143bcac0132d01a8c1e43/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:21.815862 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6965f613-0aca-456b-a504-7e51cd6ad655 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc2eab963eae48158a0f623f14731142, user_id=2160debbd32143bcac0132d01a8c1e43, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:21.818745 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6965f613-0aca-456b-a504-7e51cd6ad655 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:21.832579 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 935/1873] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:17:21 2022] PUT /identity/v3/projects/bc2eab963eae48158a0f623f14731142/users/2160debbd32143bcac0132d01a8c1e43/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:21.837080 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf64fb8-cd43-462c-9976-b0f1deb61066 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:21.847820 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf64fb8-cd43-462c-9976-b0f1deb61066 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:21.907590 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f6e5a5-6e1d-4fdf-8fff-104bc6189e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:21.908205 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f6e5a5-6e1d-4fdf-8fff-104bc6189e79 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:21.908484 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f6e5a5-6e1d-4fdf-8fff-104bc6189e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:21.908743 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f6e5a5-6e1d-4fdf-8fff-104bc6189e79 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:21.909448 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f6e5a5-6e1d-4fdf-8fff-104bc6189e79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:21.911169 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f6e5a5-6e1d-4fdf-8fff-104bc6189e79 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:21.921832 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 939/1874] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:17:21 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:17:21.926978 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6965f613-0aca-456b-a504-7e51cd6ad655 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:21.932948 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:21.939148 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6965f613-0aca-456b-a504-7e51cd6ad655 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:21.945300 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310a2a0c-3f17-4bb6-bd68-4de45fab1429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:21.945996 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310a2a0c-3f17-4bb6-bd68-4de45fab1429 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:21.946271 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310a2a0c-3f17-4bb6-bd68-4de45fab1429 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:21.946537 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310a2a0c-3f17-4bb6-bd68-4de45fab1429 admin admin] PATH_INFO: `/v3/projects/bc2eab963eae48158a0f623f14731142/users/2160debbd32143bcac0132d01a8c1e43/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:21.948509 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310a2a0c-3f17-4bb6-bd68-4de45fab1429 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc2eab963eae48158a0f623f14731142, user_id=2160debbd32143bcac0132d01a8c1e43, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:21.951197 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310a2a0c-3f17-4bb6-bd68-4de45fab1429 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:21.961732 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 936/1875] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:17:21 2022] PUT /identity/v3/projects/bc2eab963eae48158a0f623f14731142/users/2160debbd32143bcac0132d01a8c1e43/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:21.972405 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c847b5-e8ce-4ac6-be75-2cc22bc02dba None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:21.972687 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c847b5-e8ce-4ac6-be75-2cc22bc02dba None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:21.972994 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c847b5-e8ce-4ac6-be75-2cc22bc02dba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:22.002728 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-11c847b5-e8ce-4ac6-be75-2cc22bc02dba None None] MFA Rules not processed for user `2160debbd32143bcac0132d01a8c1e43`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:22.003956 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-11c847b5-e8ce-4ac6-be75-2cc22bc02dba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:22.044532 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:22.079837 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 940/1876] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:17:21 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:22.091902 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310a2a0c-3f17-4bb6-bd68-4de45fab1429 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:22.093687 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-310a2a0c-3f17-4bb6-bd68-4de45fab1429 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:22.157775 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310a2a0c-3f17-4bb6-bd68-4de45fab1429 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:22.161803 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea30b57-047e-4541-905d-c8b476af4ff9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:22.162448 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea30b57-047e-4541-905d-c8b476af4ff9 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:22.162704 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea30b57-047e-4541-905d-c8b476af4ff9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:22.162982 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea30b57-047e-4541-905d-c8b476af4ff9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:22.173163 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea30b57-047e-4541-905d-c8b476af4ff9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:22.174741 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea30b57-047e-4541-905d-c8b476af4ff9 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:22.184643 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 937/1877] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:17:22 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:17:22.638812 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c847b5-e8ce-4ac6-be75-2cc22bc02dba None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:22.719638 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c847b5-e8ce-4ac6-be75-2cc22bc02dba None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:22.725152 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7e5ee6-7bcd-4f7d-99b1-98fdd76bb432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:22.725783 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7e5ee6-7bcd-4f7d-99b1-98fdd76bb432 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:22.726055 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7e5ee6-7bcd-4f7d-99b1-98fdd76bb432 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:22.726316 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7e5ee6-7bcd-4f7d-99b1-98fdd76bb432 admin admin] PATH_INFO: `/v3/users/c8447ea02f4444b2bd165dbb30b3031b` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:22.727460 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7e5ee6-7bcd-4f7d-99b1-98fdd76bb432 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8447ea02f4444b2bd165dbb30b3031b)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:22.729514 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7e5ee6-7bcd-4f7d-99b1-98fdd76bb432 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:22.788691 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1a7e5ee6-7bcd-4f7d-99b1-98fdd76bb432 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8447ea02f4444b2bd165dbb30b3031b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:22.788691 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1a7e5ee6-7bcd-4f7d-99b1-98fdd76bb432 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8447ea02f4444b2bd165dbb30b3031b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:22.798896 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1a7e5ee6-7bcd-4f7d-99b1-98fdd76bb432 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8447ea02f4444b2bd165dbb30b3031b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:22.815041 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1a7e5ee6-7bcd-4f7d-99b1-98fdd76bb432 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8447ea02f4444b2bd165dbb30b3031b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:22.830695 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1a7e5ee6-7bcd-4f7d-99b1-98fdd76bb432 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8447ea02f4444b2bd165dbb30b3031b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:22.837469 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 941/1878] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:17:22 2022] DELETE /identity/v3/users/c8447ea02f4444b2bd165dbb30b3031b => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:22.842067 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb97a93a-0134-4cbb-bea2-60d911963415 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:22.842575 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb97a93a-0134-4cbb-bea2-60d911963415 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:22.842575 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb97a93a-0134-4cbb-bea2-60d911963415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:22.868132 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-bb97a93a-0134-4cbb-bea2-60d911963415 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:22.870444 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-bb97a93a-0134-4cbb-bea2-60d911963415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:22.950076 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 938/1879] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:17:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:22.960674 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7e5ee6-7bcd-4f7d-99b1-98fdd76bb432 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:22.962405 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-1a7e5ee6-7bcd-4f7d-99b1-98fdd76bb432 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:23.030655 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7e5ee6-7bcd-4f7d-99b1-98fdd76bb432 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:23.036746 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867d9d32-271f-4eed-8f20-4004ac647873 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:23.037411 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867d9d32-271f-4eed-8f20-4004ac647873 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:23.037750 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867d9d32-271f-4eed-8f20-4004ac647873 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:23.037993 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867d9d32-271f-4eed-8f20-4004ac647873 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:23.045848 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:23.048511 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867d9d32-271f-4eed-8f20-4004ac647873 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:23.051030 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867d9d32-271f-4eed-8f20-4004ac647873 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:23.062614 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 942/1880] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:17:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:17:23.362179 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb97a93a-0134-4cbb-bea2-60d911963415 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:23.365642 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-bb97a93a-0134-4cbb-bea2-60d911963415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:23.384912 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb97a93a-0134-4cbb-bea2-60d911963415 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:23.394229 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf42267-0797-4d93-80cb-012026d6bfc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:23.394892 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf42267-0797-4d93-80cb-012026d6bfc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:23.395194 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf42267-0797-4d93-80cb-012026d6bfc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:23.395466 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf42267-0797-4d93-80cb-012026d6bfc0 admin admin] PATH_INFO: `/v3/projects/8346a87933d34cbfae0857f49eb8e959` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:23.397244 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf42267-0797-4d93-80cb-012026d6bfc0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8346a87933d34cbfae0857f49eb8e959)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:23.399566 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf42267-0797-4d93-80cb-012026d6bfc0 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:23.467078 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ebf42267-0797-4d93-80cb-012026d6bfc0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8346a87933d34cbfae0857f49eb8e959', 'request_id': 'req-ebf42267-0797-4d93-80cb-012026d6bfc0'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:23.473546 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ebf42267-0797-4d93-80cb-012026d6bfc0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8346a87933d34cbfae0857f49eb8e959', 'request_id': 'req-ebf42267-0797-4d93-80cb-012026d6bfc0'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:23.482942 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ebf42267-0797-4d93-80cb-012026d6bfc0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8346a87933d34cbfae0857f49eb8e959', 'request_id': 'req-ebf42267-0797-4d93-80cb-012026d6bfc0'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:23.495253 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ebf42267-0797-4d93-80cb-012026d6bfc0 admin admin] The token cache is being invalidate because project 8346a87933d34cbfae0857f49eb8e959 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:17:23.495511 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ebf42267-0797-4d93-80cb-012026d6bfc0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:23.496153 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ebf42267-0797-4d93-80cb-012026d6bfc0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:23.497704 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 939/1881] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:17:23 2022] DELETE /identity/v3/projects/8346a87933d34cbfae0857f49eb8e959 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:23.501835 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867d9d32-271f-4eed-8f20-4004ac647873 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:23.512160 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867d9d32-271f-4eed-8f20-4004ac647873 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:23.583107 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d382838-cb0c-4ae9-abcc-1a1c664960a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:23.583693 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d382838-cb0c-4ae9-abcc-1a1c664960a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:23.583960 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d382838-cb0c-4ae9-abcc-1a1c664960a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:23.584226 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d382838-cb0c-4ae9-abcc-1a1c664960a5 admin admin] PATH_INFO: `/v3/users/2160debbd32143bcac0132d01a8c1e43` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:23.585524 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d382838-cb0c-4ae9-abcc-1a1c664960a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2160debbd32143bcac0132d01a8c1e43)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:23.587484 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d382838-cb0c-4ae9-abcc-1a1c664960a5 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:23.642120 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-4d382838-cb0c-4ae9-abcc-1a1c664960a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2160debbd32143bcac0132d01a8c1e43'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:23.642791 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-4d382838-cb0c-4ae9-abcc-1a1c664960a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2160debbd32143bcac0132d01a8c1e43'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:23.650580 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-4d382838-cb0c-4ae9-abcc-1a1c664960a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2160debbd32143bcac0132d01a8c1e43'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:23.663015 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-4d382838-cb0c-4ae9-abcc-1a1c664960a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2160debbd32143bcac0132d01a8c1e43'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:23.674284 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-4d382838-cb0c-4ae9-abcc-1a1c664960a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2160debbd32143bcac0132d01a8c1e43'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:23.681312 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 943/1882] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:17:23 2022] DELETE /identity/v3/users/2160debbd32143bcac0132d01a8c1e43 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:24.028588 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf42267-0797-4d93-80cb-012026d6bfc0 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:24.030659 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ebf42267-0797-4d93-80cb-012026d6bfc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:24.054409 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf42267-0797-4d93-80cb-012026d6bfc0 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:24.076978 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:24.127580 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d66d75-cfce-42b4-bcf5-cadcfe30df59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:24.128359 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d66d75-cfce-42b4-bcf5-cadcfe30df59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:24.128634 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d66d75-cfce-42b4-bcf5-cadcfe30df59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:24.128938 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d66d75-cfce-42b4-bcf5-cadcfe30df59 admin admin] PATH_INFO: `/v3/projects/bc2eab963eae48158a0f623f14731142` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:24.130205 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d66d75-cfce-42b4-bcf5-cadcfe30df59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc2eab963eae48158a0f623f14731142)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:24.132793 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d66d75-cfce-42b4-bcf5-cadcfe30df59 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:24.185685 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e2d66d75-cfce-42b4-bcf5-cadcfe30df59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc2eab963eae48158a0f623f14731142', 'request_id': 'req-e2d66d75-cfce-42b4-bcf5-cadcfe30df59'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:24.191742 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e2d66d75-cfce-42b4-bcf5-cadcfe30df59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc2eab963eae48158a0f623f14731142', 'request_id': 'req-e2d66d75-cfce-42b4-bcf5-cadcfe30df59'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:24.201633 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e2d66d75-cfce-42b4-bcf5-cadcfe30df59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc2eab963eae48158a0f623f14731142', 'request_id': 'req-e2d66d75-cfce-42b4-bcf5-cadcfe30df59'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:24.215525 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e2d66d75-cfce-42b4-bcf5-cadcfe30df59 admin admin] The token cache is being invalidate because project bc2eab963eae48158a0f623f14731142 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:17:24.215772 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e2d66d75-cfce-42b4-bcf5-cadcfe30df59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:24.216477 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e2d66d75-cfce-42b4-bcf5-cadcfe30df59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:24.218092 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 940/1883] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:17:24 2022] DELETE /identity/v3/projects/bc2eab963eae48158a0f623f14731142 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:17:24.224669 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7180ed07-c2e4-4a3a-bf89-75a131ee48b0 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:24.225236 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7180ed07-c2e4-4a3a-bf89-75a131ee48b0 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:24.225236 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7180ed07-c2e4-4a3a-bf89-75a131ee48b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:24.251974 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-7180ed07-c2e4-4a3a-bf89-75a131ee48b0 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:24.253310 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7180ed07-c2e4-4a3a-bf89-75a131ee48b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:24.326843 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 944/1884] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:17:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:17:24.334756 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a2a08f-c482-4b5f-bc45-6181ed6daedb None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:24.335027 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a2a08f-c482-4b5f-bc45-6181ed6daedb None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:24.335294 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a2a08f-c482-4b5f-bc45-6181ed6daedb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:24.363243 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-38a2a08f-c482-4b5f-bc45-6181ed6daedb None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:24.364536 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-38a2a08f-c482-4b5f-bc45-6181ed6daedb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:24.373478 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 941/1885] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:17:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:24.378066 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7180ed07-c2e4-4a3a-bf89-75a131ee48b0 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:24.387990 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7180ed07-c2e4-4a3a-bf89-75a131ee48b0 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:24.392880 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59251d7-4476-43fc-a7eb-179d95c78582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:24.393490 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59251d7-4476-43fc-a7eb-179d95c78582 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:24.393765 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59251d7-4476-43fc-a7eb-179d95c78582 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:24.394035 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59251d7-4476-43fc-a7eb-179d95c78582 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:24.394687 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59251d7-4476-43fc-a7eb-179d95c78582 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:17:24.394985 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59251d7-4476-43fc-a7eb-179d95c78582 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:24.396674 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59251d7-4476-43fc-a7eb-179d95c78582 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:24.409280 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 945/1886] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 6 16:17:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:17:24.414014 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a2a08f-c482-4b5f-bc45-6181ed6daedb None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:24.423489 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a2a08f-c482-4b5f-bc45-6181ed6daedb None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:24.428906 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64bb0f5-6b7b-4d7a-a34b-36f19ff00e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:24.429505 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64bb0f5-6b7b-4d7a-a34b-36f19ff00e1c admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:24.429834 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64bb0f5-6b7b-4d7a-a34b-36f19ff00e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:24.430189 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64bb0f5-6b7b-4d7a-a34b-36f19ff00e1c admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:24.430919 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64bb0f5-6b7b-4d7a-a34b-36f19ff00e1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:24.433011 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64bb0f5-6b7b-4d7a-a34b-36f19ff00e1c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:24.468872 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59251d7-4476-43fc-a7eb-179d95c78582 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:24.471122 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c59251d7-4476-43fc-a7eb-179d95c78582 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:24.479346 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:24.482430 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 942/1887] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:17:24 2022] POST /identity/v3/projects => generated 379 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:17:24.487101 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64bb0f5-6b7b-4d7a-a34b-36f19ff00e1c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:24.488378 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59251d7-4476-43fc-a7eb-179d95c78582 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:24.555511 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64bb0f5-6b7b-4d7a-a34b-36f19ff00e1c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:24.562382 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cdc290-2ad7-44bd-9d2b-3ca0aad6c2e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:24.563546 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cdc290-2ad7-44bd-9d2b-3ca0aad6c2e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:24.563979 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cdc290-2ad7-44bd-9d2b-3ca0aad6c2e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:24.564396 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cdc290-2ad7-44bd-9d2b-3ca0aad6c2e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:24.565446 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cdc290-2ad7-44bd-9d2b-3ca0aad6c2e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:24.566337 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2ce508-9b2d-4be2-840f-2dac2eb81bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:24.567013 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2ce508-9b2d-4be2-840f-2dac2eb81bdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:24.567473 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2ce508-9b2d-4be2-840f-2dac2eb81bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:24.568288 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2ce508-9b2d-4be2-840f-2dac2eb81bdb admin admin] PATH_INFO: `/v3/users/c1a01b383dce471c87424cde937d74c8` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:24.568541 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cdc290-2ad7-44bd-9d2b-3ca0aad6c2e4 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:24.569117 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2ce508-9b2d-4be2-840f-2dac2eb81bdb admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1a01b383dce471c87424cde937d74c8)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:24.571004 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2ce508-9b2d-4be2-840f-2dac2eb81bdb admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:24.596303 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 943/1888] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:17:24 2022] POST /identity/v3/users => generated 290 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:17:24.600392 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cdc290-2ad7-44bd-9d2b-3ca0aad6c2e4 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:24.614484 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cdc290-2ad7-44bd-9d2b-3ca0aad6c2e4 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:24.621950 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfdbecc-d07f-4166-af4b-f4fce130ab65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:24.623309 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfdbecc-d07f-4166-af4b-f4fce130ab65 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:24.623428 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfdbecc-d07f-4166-af4b-f4fce130ab65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:24.623707 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfdbecc-d07f-4166-af4b-f4fce130ab65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:24.624530 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfdbecc-d07f-4166-af4b-f4fce130ab65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:24.626603 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfdbecc-d07f-4166-af4b-f4fce130ab65 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:24.629606 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:24.638078 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1d2ce508-9b2d-4be2-840f-2dac2eb81bdb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1a01b383dce471c87424cde937d74c8'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:24.638949 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1d2ce508-9b2d-4be2-840f-2dac2eb81bdb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1a01b383dce471c87424cde937d74c8'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:24.641483 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 944/1889] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:17:24 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:17:24.660004 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfdbecc-d07f-4166-af4b-f4fce130ab65 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:24.668254 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1d2ce508-9b2d-4be2-840f-2dac2eb81bdb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1a01b383dce471c87424cde937d74c8'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:24.675398 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfdbecc-d07f-4166-af4b-f4fce130ab65 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:24.685773 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4d35a1-b953-476e-8290-f5e08eba69b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:24.687119 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4d35a1-b953-476e-8290-f5e08eba69b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:24.687644 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4d35a1-b953-476e-8290-f5e08eba69b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:24.688071 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4d35a1-b953-476e-8290-f5e08eba69b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:24.688618 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1d2ce508-9b2d-4be2-840f-2dac2eb81bdb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1a01b383dce471c87424cde937d74c8'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:24.688736 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4d35a1-b953-476e-8290-f5e08eba69b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:24.690450 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4d35a1-b953-476e-8290-f5e08eba69b9 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:24.701261 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1d2ce508-9b2d-4be2-840f-2dac2eb81bdb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1a01b383dce471c87424cde937d74c8'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:24.706773 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 945/1890] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:17:24 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 16:17:24.711371 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 946/1891] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:17:24 2022] DELETE /identity/v3/users/c1a01b383dce471c87424cde937d74c8 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:17:24.711931 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4d35a1-b953-476e-8290-f5e08eba69b9 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:24.717283 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ea4d35a1-b953-476e-8290-f5e08eba69b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:24.717461 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1dd356-891c-41da-8d4d-95031b123218 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:24.717618 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1dd356-891c-41da-8d4d-95031b123218 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:24.717924 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1dd356-891c-41da-8d4d-95031b123218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:24.735264 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4d35a1-b953-476e-8290-f5e08eba69b9 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:24.740899 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-8d1dd356-891c-41da-8d4d-95031b123218 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:24.742126 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8d1dd356-891c-41da-8d4d-95031b123218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:24.792711 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac6bb19-354c-4f14-b3db-78c12485d0e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:24.793408 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac6bb19-354c-4f14-b3db-78c12485d0e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:24.793690 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac6bb19-354c-4f14-b3db-78c12485d0e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:24.793965 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac6bb19-354c-4f14-b3db-78c12485d0e6 admin admin] PATH_INFO: `/v3/projects/c6e56a2ac043457b81a0c3d01f8dab2c/users/f5f9e500946547f7a00752350f10bbaf/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:24.809663 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac6bb19-354c-4f14-b3db-78c12485d0e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6e56a2ac043457b81a0c3d01f8dab2c, user_id=f5f9e500946547f7a00752350f10bbaf, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:24.810672 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 947/1892] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:17:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:24.812672 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac6bb19-354c-4f14-b3db-78c12485d0e6 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:24.822337 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1dd356-891c-41da-8d4d-95031b123218 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:24.823978 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8d1dd356-891c-41da-8d4d-95031b123218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:24.830450 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 946/1893] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:17:24 2022] PUT /identity/v3/projects/c6e56a2ac043457b81a0c3d01f8dab2c/users/f5f9e500946547f7a00752350f10bbaf/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:24.844207 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3606e2-d50c-4312-a93e-6593ce42f704 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:24.844457 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3606e2-d50c-4312-a93e-6593ce42f704 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:24.844727 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3606e2-d50c-4312-a93e-6593ce42f704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:24.852968 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:24.880024 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-0e3606e2-d50c-4312-a93e-6593ce42f704 None None] MFA Rules not processed for user `f5f9e500946547f7a00752350f10bbaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:24.881727 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0e3606e2-d50c-4312-a93e-6593ce42f704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:24.890803 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1dd356-891c-41da-8d4d-95031b123218 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:24.940230 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a382b9e8-aa99-4eda-8858-81d6cd444855 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:24.940958 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a382b9e8-aa99-4eda-8858-81d6cd444855 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:24.941237 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a382b9e8-aa99-4eda-8858-81d6cd444855 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:24.941517 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a382b9e8-aa99-4eda-8858-81d6cd444855 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:24.955267 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 947/1894] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:17:24 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:17:24.960016 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3606e2-d50c-4312-a93e-6593ce42f704 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:25.009815 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a382b9e8-aa99-4eda-8858-81d6cd444855 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:25.011528 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a382b9e8-aa99-4eda-8858-81d6cd444855 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:25.021776 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 948/1895] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:17:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:17:25.023537 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:25.027815 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3606e2-d50c-4312-a93e-6593ce42f704 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:25.034332 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f001c883-523c-410c-8778-8549df450f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:25.034941 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f001c883-523c-410c-8778-8549df450f0d admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:25.035213 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f001c883-523c-410c-8778-8549df450f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:25.035484 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f001c883-523c-410c-8778-8549df450f0d admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:25.036270 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f001c883-523c-410c-8778-8549df450f0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:25.038457 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f001c883-523c-410c-8778-8549df450f0d admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:25.068083 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 948/1896] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:17:24 2022] POST /identity/v3/projects => generated 381 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:17:25.072252 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a382b9e8-aa99-4eda-8858-81d6cd444855 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:25.083482 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a382b9e8-aa99-4eda-8858-81d6cd444855 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:25.186523 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad13ca37-b455-40f3-94d1-8b22277fb060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:25.188150 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad13ca37-b455-40f3-94d1-8b22277fb060 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:25.189262 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad13ca37-b455-40f3-94d1-8b22277fb060 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:25.189610 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad13ca37-b455-40f3-94d1-8b22277fb060 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:25.190323 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad13ca37-b455-40f3-94d1-8b22277fb060 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:25.194620 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad13ca37-b455-40f3-94d1-8b22277fb060 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:25.244859 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 949/1897] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:17:25 2022] POST /identity/v3/users => generated 297 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:17:25.248931 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f001c883-523c-410c-8778-8549df450f0d admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:25.265849 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f001c883-523c-410c-8778-8549df450f0d admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:25.275533 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93940c15-8505-4d33-a5ce-f7ecdaafe1ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:25.276003 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93940c15-8505-4d33-a5ce-f7ecdaafe1ad admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:25.276358 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93940c15-8505-4d33-a5ce-f7ecdaafe1ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:25.276623 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93940c15-8505-4d33-a5ce-f7ecdaafe1ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:25.277837 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93940c15-8505-4d33-a5ce-f7ecdaafe1ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:25.290406 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93940c15-8505-4d33-a5ce-f7ecdaafe1ad admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:25.305415 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 949/1898] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:17:25 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 16:17:25.310481 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad13ca37-b455-40f3-94d1-8b22277fb060 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:25.323148 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad13ca37-b455-40f3-94d1-8b22277fb060 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:25.335179 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abe9a3a-8a92-4a6a-bf46-1fc75601be1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:25.335876 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abe9a3a-8a92-4a6a-bf46-1fc75601be1b admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:25.336154 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abe9a3a-8a92-4a6a-bf46-1fc75601be1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:25.336428 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abe9a3a-8a92-4a6a-bf46-1fc75601be1b admin admin] PATH_INFO: `/v3/projects/b6dea956937c4e6d9955c0499d4ff069/users/7e5420775faa44bc88a817caee3d5a4c/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:25.353855 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abe9a3a-8a92-4a6a-bf46-1fc75601be1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6dea956937c4e6d9955c0499d4ff069, user_id=7e5420775faa44bc88a817caee3d5a4c, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:25.358483 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abe9a3a-8a92-4a6a-bf46-1fc75601be1b admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:25.361727 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:25.383066 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 950/1899] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:17:25 2022] PUT /identity/v3/projects/b6dea956937c4e6d9955c0499d4ff069/users/7e5420775faa44bc88a817caee3d5a4c/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:25.388986 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93940c15-8505-4d33-a5ce-f7ecdaafe1ad admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:25.403334 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93940c15-8505-4d33-a5ce-f7ecdaafe1ad admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:25.404635 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abe9a3a-8a92-4a6a-bf46-1fc75601be1b admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:25.407180 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7abe9a3a-8a92-4a6a-bf46-1fc75601be1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:25.489410 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbc546c-b5d8-4c8c-9055-ccec9f207670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:25.490193 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbc546c-b5d8-4c8c-9055-ccec9f207670 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:25.490472 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbc546c-b5d8-4c8c-9055-ccec9f207670 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:25.490746 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbc546c-b5d8-4c8c-9055-ccec9f207670 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:25.491467 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbc546c-b5d8-4c8c-9055-ccec9f207670 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:25.492240 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abe9a3a-8a92-4a6a-bf46-1fc75601be1b admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:25.493376 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbc546c-b5d8-4c8c-9055-ccec9f207670 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:25.497897 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cf8b79-c59a-4c9b-81c2-7cf0e78d494c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:25.498542 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf8b79-c59a-4c9b-81c2-7cf0e78d494c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:25.498805 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf8b79-c59a-4c9b-81c2-7cf0e78d494c admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:25.499065 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf8b79-c59a-4c9b-81c2-7cf0e78d494c admin admin] PATH_INFO: `/v3/projects/7dcd024ced6a46bd9541eb3de7ce5a20` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:25.500199 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cf8b79-c59a-4c9b-81c2-7cf0e78d494c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7dcd024ced6a46bd9541eb3de7ce5a20)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:25.502264 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cf8b79-c59a-4c9b-81c2-7cf0e78d494c admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:25.510319 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 950/1900] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:17:25 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:17:25.511486 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbc546c-b5d8-4c8c-9055-ccec9f207670 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:25.525841 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbc546c-b5d8-4c8c-9055-ccec9f207670 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:25.533245 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d961cd6a-d263-456b-a2fb-805cf26eec55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:25.533954 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d961cd6a-d263-456b-a2fb-805cf26eec55 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:25.534230 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d961cd6a-d263-456b-a2fb-805cf26eec55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:25.534510 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d961cd6a-d263-456b-a2fb-805cf26eec55 admin admin] PATH_INFO: `/v3/projects/b6dea956937c4e6d9955c0499d4ff069/users/7e5420775faa44bc88a817caee3d5a4c/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:25.536910 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d961cd6a-d263-456b-a2fb-805cf26eec55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6dea956937c4e6d9955c0499d4ff069, user_id=7e5420775faa44bc88a817caee3d5a4c, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:25.539757 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d961cd6a-d263-456b-a2fb-805cf26eec55 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:25.552160 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 951/1901] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:17:25 2022] PUT /identity/v3/projects/b6dea956937c4e6d9955c0499d4ff069/users/7e5420775faa44bc88a817caee3d5a4c/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:17:25.559304 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-13cf8b79-c59a-4c9b-81c2-7cf0e78d494c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7dcd024ced6a46bd9541eb3de7ce5a20', 'request_id': 'req-13cf8b79-c59a-4c9b-81c2-7cf0e78d494c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:25.563606 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9bea60-1f16-4f2b-aabb-79f7b9f831fa None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:25.563606 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9bea60-1f16-4f2b-aabb-79f7b9f831fa None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:25.563606 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9bea60-1f16-4f2b-aabb-79f7b9f831fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:25.568068 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-13cf8b79-c59a-4c9b-81c2-7cf0e78d494c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7dcd024ced6a46bd9541eb3de7ce5a20', 'request_id': 'req-13cf8b79-c59a-4c9b-81c2-7cf0e78d494c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:25.575058 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-13cf8b79-c59a-4c9b-81c2-7cf0e78d494c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7dcd024ced6a46bd9541eb3de7ce5a20', 'request_id': 'req-13cf8b79-c59a-4c9b-81c2-7cf0e78d494c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:25.587113 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-13cf8b79-c59a-4c9b-81c2-7cf0e78d494c admin admin] The token cache is being invalidate because project 7dcd024ced6a46bd9541eb3de7ce5a20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:17:25.587398 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-13cf8b79-c59a-4c9b-81c2-7cf0e78d494c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:25.588120 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-13cf8b79-c59a-4c9b-81c2-7cf0e78d494c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:25.589766 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 951/1902] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:17:25 2022] DELETE /identity/v3/projects/7dcd024ced6a46bd9541eb3de7ce5a20 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:25.591562 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-3e9bea60-1f16-4f2b-aabb-79f7b9f831fa None None] MFA Rules not processed for user `7e5420775faa44bc88a817caee3d5a4c`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:25.592942 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3e9bea60-1f16-4f2b-aabb-79f7b9f831fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:25.594607 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a02b2-7385-41e7-8280-59fbbe14ece2 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:25.594884 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a02b2-7385-41e7-8280-59fbbe14ece2 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:25.595155 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a02b2-7385-41e7-8280-59fbbe14ece2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:25.615615 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-1f9a02b2-7385-41e7-8280-59fbbe14ece2 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:25.616714 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-1f9a02b2-7385-41e7-8280-59fbbe14ece2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:25.670325 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 952/1903] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:17:25 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:17:25.688178 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9bea60-1f16-4f2b-aabb-79f7b9f831fa None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:25.690165 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3e9bea60-1f16-4f2b-aabb-79f7b9f831fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:25.705918 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 952/1904] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:17:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:25.708082 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8075890-f224-4a72-b44e-98766be3704b None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:25.708372 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8075890-f224-4a72-b44e-98766be3704b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:25.708650 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8075890-f224-4a72-b44e-98766be3704b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:25.731764 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-e8075890-f224-4a72-b44e-98766be3704b None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:25.733229 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e8075890-f224-4a72-b44e-98766be3704b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:25.745790 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 953/1905] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:17:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:17:25.749109 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:25.750582 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8075890-f224-4a72-b44e-98766be3704b None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:25.764717 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8075890-f224-4a72-b44e-98766be3704b None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:25.769361 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9bea60-1f16-4f2b-aabb-79f7b9f831fa None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:25.774854 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e511e275-2117-4027-b705-9cbef941d337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:25.775622 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e511e275-2117-4027-b705-9cbef941d337 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:25.776514 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac3ed5f-477a-4348-86fd-31ae29668c42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:25.777589 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3ed5f-477a-4348-86fd-31ae29668c42 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:25.778200 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3ed5f-477a-4348-86fd-31ae29668c42 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:25.778552 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3ed5f-477a-4348-86fd-31ae29668c42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:25.779853 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e511e275-2117-4027-b705-9cbef941d337 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:25.780294 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e511e275-2117-4027-b705-9cbef941d337 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:25.781624 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e511e275-2117-4027-b705-9cbef941d337 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:17:25.781841 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e511e275-2117-4027-b705-9cbef941d337 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:25.785888 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e511e275-2117-4027-b705-9cbef941d337 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:25.801960 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac3ed5f-477a-4348-86fd-31ae29668c42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:25.803725 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac3ed5f-477a-4348-86fd-31ae29668c42 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:25.809057 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 954/1906] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 6 16:17:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:17:25.811672 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e511e275-2117-4027-b705-9cbef941d337 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:25.818365 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 953/1907] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:17:25 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:17:25.824074 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:25.828174 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e511e275-2117-4027-b705-9cbef941d337 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:25.836077 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b44bd3-85a0-47e9-ab02-36db598d8d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:25.837675 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b44bd3-85a0-47e9-ab02-36db598d8d1c admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:25.838150 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b44bd3-85a0-47e9-ab02-36db598d8d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:25.838480 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b44bd3-85a0-47e9-ab02-36db598d8d1c admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:25.839375 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b44bd3-85a0-47e9-ab02-36db598d8d1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:25.841451 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b44bd3-85a0-47e9-ab02-36db598d8d1c admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:25.887611 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 955/1908] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:17:25 2022] POST /identity/v3/projects => generated 359 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:17:25.892293 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac3ed5f-477a-4348-86fd-31ae29668c42 service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:25.905177 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac3ed5f-477a-4348-86fd-31ae29668c42 service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:25.984330 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b5b164-0d8c-4dab-87fd-35e44e7b6efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:25.984991 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5b164-0d8c-4dab-87fd-35e44e7b6efa admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:25.985273 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5b164-0d8c-4dab-87fd-35e44e7b6efa admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:25.985545 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5b164-0d8c-4dab-87fd-35e44e7b6efa admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:25.986223 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b5b164-0d8c-4dab-87fd-35e44e7b6efa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:25.988166 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b5b164-0d8c-4dab-87fd-35e44e7b6efa admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:26.019315 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 954/1909] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:17:25 2022] POST /identity/v3/users => generated 280 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:17:26.023703 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b44bd3-85a0-47e9-ab02-36db598d8d1c admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:26.034230 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b44bd3-85a0-47e9-ab02-36db598d8d1c admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:26.039901 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3725fc65-10a0-4e3b-9f93-93b51d5900d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:26.040616 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3725fc65-10a0-4e3b-9f93-93b51d5900d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:26.040969 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3725fc65-10a0-4e3b-9f93-93b51d5900d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:26.041342 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3725fc65-10a0-4e3b-9f93-93b51d5900d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:26.042195 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3725fc65-10a0-4e3b-9f93-93b51d5900d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:26.044256 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3725fc65-10a0-4e3b-9f93-93b51d5900d6 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:26.058618 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 956/1910] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:17:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:17:26.059923 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b5b164-0d8c-4dab-87fd-35e44e7b6efa admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:26.070627 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b5b164-0d8c-4dab-87fd-35e44e7b6efa admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:26.076491 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfea99f2-f49a-4273-a862-37fa408288f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:26.077160 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfea99f2-f49a-4273-a862-37fa408288f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:26.077436 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfea99f2-f49a-4273-a862-37fa408288f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:26.077687 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfea99f2-f49a-4273-a862-37fa408288f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:26.078338 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfea99f2-f49a-4273-a862-37fa408288f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:26.079973 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfea99f2-f49a-4273-a862-37fa408288f2 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:26.092123 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 955/1911] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:17:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:17:26.096584 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3725fc65-10a0-4e3b-9f93-93b51d5900d6 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:26.107564 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3725fc65-10a0-4e3b-9f93-93b51d5900d6 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:26.114737 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eedcc5-1fce-4d13-a1b7-3110c1a79f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:26.115630 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eedcc5-1fce-4d13-a1b7-3110c1a79f7e admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:26.115917 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eedcc5-1fce-4d13-a1b7-3110c1a79f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:26.116194 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eedcc5-1fce-4d13-a1b7-3110c1a79f7e admin admin] PATH_INFO: `/v3/projects/9693fb70c24a4220b2acf6446aa87a70/users/6fce4031fe294d16ad27f0cde9c8120b/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:26.169719 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6eedcc5-1fce-4d13-a1b7-3110c1a79f7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9693fb70c24a4220b2acf6446aa87a70, user_id=6fce4031fe294d16ad27f0cde9c8120b, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:26.182102 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6eedcc5-1fce-4d13-a1b7-3110c1a79f7e admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:26.216135 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 957/1912] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:17:26 2022] PUT /identity/v3/projects/9693fb70c24a4220b2acf6446aa87a70/users/6fce4031fe294d16ad27f0cde9c8120b/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:26.227336 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d0f194-7237-4ece-af00-09f20d4fb168 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:26.227609 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d0f194-7237-4ece-af00-09f20d4fb168 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:26.228120 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d0f194-7237-4ece-af00-09f20d4fb168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:26.303329 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-73d0f194-7237-4ece-af00-09f20d4fb168 None None] MFA Rules not processed for user `6fce4031fe294d16ad27f0cde9c8120b`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:26.305838 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-73d0f194-7237-4ece-af00-09f20d4fb168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:26.375697 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eedcc5-1fce-4d13-a1b7-3110c1a79f7e admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:26.510795 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 956/1913] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:17:26 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:26.571020 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eedcc5-1fce-4d13-a1b7-3110c1a79f7e admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:26.581459 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6a034b-7de3-47c6-bc7b-5c7a138eea48 tempest-VolumeQuotaClassesTest-1990960567 tempest-VolumeQuotaClassesTest-1990960567-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e5420775faa44bc88a817caee3d5a4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6dea956937c4e6d9955c0499d4ff069', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:26.581907 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6a034b-7de3-47c6-bc7b-5c7a138eea48 tempest-VolumeQuotaClassesTest-1990960567 tempest-VolumeQuotaClassesTest-1990960567-project-admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:26.582233 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6a034b-7de3-47c6-bc7b-5c7a138eea48 tempest-VolumeQuotaClassesTest-1990960567 tempest-VolumeQuotaClassesTest-1990960567-project-admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:26.582386 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6a034b-7de3-47c6-bc7b-5c7a138eea48 tempest-VolumeQuotaClassesTest-1990960567 tempest-VolumeQuotaClassesTest-1990960567-project-admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:26.583169 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6a034b-7de3-47c6-bc7b-5c7a138eea48 tempest-VolumeQuotaClassesTest-1990960567 tempest-VolumeQuotaClassesTest-1990960567-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:17:26.583475 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6a034b-7de3-47c6-bc7b-5c7a138eea48 tempest-VolumeQuotaClassesTest-1990960567 tempest-VolumeQuotaClassesTest-1990960567-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:26.585322 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6a034b-7de3-47c6-bc7b-5c7a138eea48 tempest-VolumeQuotaClassesTest-1990960567 tempest-VolumeQuotaClassesTest-1990960567-project-admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:26.591287 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d0f194-7237-4ece-af00-09f20d4fb168 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:26.593564 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-73d0f194-7237-4ece-af00-09f20d4fb168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:26.604490 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 958/1914] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 6 16:17:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:17:26.605456 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6a034b-7de3-47c6-bc7b-5c7a138eea48 tempest-VolumeQuotaClassesTest-1990960567 tempest-VolumeQuotaClassesTest-1990960567-project-admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:26.693165 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:26.700347 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6a034b-7de3-47c6-bc7b-5c7a138eea48 tempest-VolumeQuotaClassesTest-1990960567 tempest-VolumeQuotaClassesTest-1990960567-project-admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:26.712635 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d0f194-7237-4ece-af00-09f20d4fb168 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:26.718457 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d49dc9-4265-4c3d-8f9c-8436bfb058ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:26.719100 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d49dc9-4265-4c3d-8f9c-8436bfb058ed service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:26.719373 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d49dc9-4265-4c3d-8f9c-8436bfb058ed service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:26.719638 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d49dc9-4265-4c3d-8f9c-8436bfb058ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:26.724454 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff067a3-ae98-4891-9224-210aac67ea0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:26.725215 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff067a3-ae98-4891-9224-210aac67ea0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:26.725518 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff067a3-ae98-4891-9224-210aac67ea0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:26.725788 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff067a3-ae98-4891-9224-210aac67ea0b admin admin] PATH_INFO: `/v3/users/7acbd98fed99463ba2f672c7619d416b` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:26.727046 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff067a3-ae98-4891-9224-210aac67ea0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7acbd98fed99463ba2f672c7619d416b)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:26.728995 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff067a3-ae98-4891-9224-210aac67ea0b admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:26.730506 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d49dc9-4265-4c3d-8f9c-8436bfb058ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:26.731979 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d49dc9-4265-4c3d-8f9c-8436bfb058ed service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:26.741488 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 959/1915] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:17:26 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:17:26.744170 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d49dc9-4265-4c3d-8f9c-8436bfb058ed service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:26.760930 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d49dc9-4265-4c3d-8f9c-8436bfb058ed service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:26.766904 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918b1390-e9e8-4bda-a75f-cccf24a67fde tempest-VolumeQuotaClassesTest-1990960567 tempest-VolumeQuotaClassesTest-1990960567-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e5420775faa44bc88a817caee3d5a4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6dea956937c4e6d9955c0499d4ff069', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:26.767721 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918b1390-e9e8-4bda-a75f-cccf24a67fde tempest-VolumeQuotaClassesTest-1990960567 tempest-VolumeQuotaClassesTest-1990960567-project-admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:26.768142 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918b1390-e9e8-4bda-a75f-cccf24a67fde tempest-VolumeQuotaClassesTest-1990960567 tempest-VolumeQuotaClassesTest-1990960567-project-admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:26.768546 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918b1390-e9e8-4bda-a75f-cccf24a67fde tempest-VolumeQuotaClassesTest-1990960567 tempest-VolumeQuotaClassesTest-1990960567-project-admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:26.769650 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918b1390-e9e8-4bda-a75f-cccf24a67fde tempest-VolumeQuotaClassesTest-1990960567 tempest-VolumeQuotaClassesTest-1990960567-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:26.773369 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918b1390-e9e8-4bda-a75f-cccf24a67fde tempest-VolumeQuotaClassesTest-1990960567 tempest-VolumeQuotaClassesTest-1990960567-project-admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:26.792021 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-0ff067a3-ae98-4891-9224-210aac67ea0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7acbd98fed99463ba2f672c7619d416b'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:26.792833 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-0ff067a3-ae98-4891-9224-210aac67ea0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7acbd98fed99463ba2f672c7619d416b'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:26.802924 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-0ff067a3-ae98-4891-9224-210aac67ea0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7acbd98fed99463ba2f672c7619d416b'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:26.816806 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 960/1916] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:17:26 2022] POST /identity/v3/projects => generated 355 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:17:26.818074 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918b1390-e9e8-4bda-a75f-cccf24a67fde tempest-VolumeQuotaClassesTest-1990960567 tempest-VolumeQuotaClassesTest-1990960567-project-admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:26.819959 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-0ff067a3-ae98-4891-9224-210aac67ea0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7acbd98fed99463ba2f672c7619d416b'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:26.821175 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-0ff067a3-ae98-4891-9224-210aac67ea0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7acbd98fed99463ba2f672c7619d416b'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:26.848205 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 957/1917] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:17:26 2022] DELETE /identity/v3/users/7acbd98fed99463ba2f672c7619d416b => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:26.851196 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff067a3-ae98-4891-9224-210aac67ea0b admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:26.853473 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0ff067a3-ae98-4891-9224-210aac67ea0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:26.863687 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:26.938181 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918b1390-e9e8-4bda-a75f-cccf24a67fde tempest-VolumeQuotaClassesTest-1990960567 tempest-VolumeQuotaClassesTest-1990960567-project-admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:26.946361 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff067a3-ae98-4891-9224-210aac67ea0b admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:26.951425 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b4a07d-b1c3-4a33-8840-e9f287dfba4d tempest-VolumeQuotaClassesTest-1990960567 tempest-VolumeQuotaClassesTest-1990960567-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e5420775faa44bc88a817caee3d5a4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6dea956937c4e6d9955c0499d4ff069', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:26.952413 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b4a07d-b1c3-4a33-8840-e9f287dfba4d tempest-VolumeQuotaClassesTest-1990960567 tempest-VolumeQuotaClassesTest-1990960567-project-admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:26.952820 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b4a07d-b1c3-4a33-8840-e9f287dfba4d tempest-VolumeQuotaClassesTest-1990960567 tempest-VolumeQuotaClassesTest-1990960567-project-admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:26.953152 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b4a07d-b1c3-4a33-8840-e9f287dfba4d tempest-VolumeQuotaClassesTest-1990960567 tempest-VolumeQuotaClassesTest-1990960567-project-admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:26.954776 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b4a07d-b1c3-4a33-8840-e9f287dfba4d tempest-VolumeQuotaClassesTest-1990960567 tempest-VolumeQuotaClassesTest-1990960567-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:17:26.955746 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b4a07d-b1c3-4a33-8840-e9f287dfba4d tempest-VolumeQuotaClassesTest-1990960567 tempest-VolumeQuotaClassesTest-1990960567-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:26.958267 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b4a07d-b1c3-4a33-8840-e9f287dfba4d tempest-VolumeQuotaClassesTest-1990960567 tempest-VolumeQuotaClassesTest-1990960567-project-admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:26.976397 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 958/1918] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 6 16:17:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:17:26.976397 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cfe8ac-2518-4ae9-a29a-ae459fff1525 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:26.976397 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cfe8ac-2518-4ae9-a29a-ae459fff1525 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:26.976397 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cfe8ac-2518-4ae9-a29a-ae459fff1525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:27.013185 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-b8cfe8ac-2518-4ae9-a29a-ae459fff1525 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:27.015123 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b8cfe8ac-2518-4ae9-a29a-ae459fff1525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:27.033051 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4b75f9-87d9-4797-8b78-4fa1494edd99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:27.033961 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4b75f9-87d9-4797-8b78-4fa1494edd99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:27.034243 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4b75f9-87d9-4797-8b78-4fa1494edd99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:27.034862 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4b75f9-87d9-4797-8b78-4fa1494edd99 admin admin] PATH_INFO: `/v3/users/6fce4031fe294d16ad27f0cde9c8120b` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:27.036772 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4b75f9-87d9-4797-8b78-4fa1494edd99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fce4031fe294d16ad27f0cde9c8120b)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:27.038771 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4b75f9-87d9-4797-8b78-4fa1494edd99 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:27.088191 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 959/1919] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:17:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:27.089194 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cfe8ac-2518-4ae9-a29a-ae459fff1525 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:27.096571 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:27.101462 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cfe8ac-2518-4ae9-a29a-ae459fff1525 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:27.104951 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-aa4b75f9-87d9-4797-8b78-4fa1494edd99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fce4031fe294d16ad27f0cde9c8120b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:27.106666 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-aa4b75f9-87d9-4797-8b78-4fa1494edd99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fce4031fe294d16ad27f0cde9c8120b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:27.110271 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2158dc29-7d39-4ae7-b1ea-af27961ef23a tempest-VolumeQuotaClassesTest-1990960567 tempest-VolumeQuotaClassesTest-1990960567-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e5420775faa44bc88a817caee3d5a4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6dea956937c4e6d9955c0499d4ff069', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:27.110990 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2158dc29-7d39-4ae7-b1ea-af27961ef23a tempest-VolumeQuotaClassesTest-1990960567 tempest-VolumeQuotaClassesTest-1990960567-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:27.111363 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2158dc29-7d39-4ae7-b1ea-af27961ef23a tempest-VolumeQuotaClassesTest-1990960567 tempest-VolumeQuotaClassesTest-1990960567-project-admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:27.112645 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2158dc29-7d39-4ae7-b1ea-af27961ef23a tempest-VolumeQuotaClassesTest-1990960567 tempest-VolumeQuotaClassesTest-1990960567-project-admin] PATH_INFO: `/v3/projects/0301253f939841e98071973c9b4b0bc5` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:27.114539 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2158dc29-7d39-4ae7-b1ea-af27961ef23a tempest-VolumeQuotaClassesTest-1990960567 tempest-VolumeQuotaClassesTest-1990960567-project-admin] RBAC: Authorizing `identity:delete_project(project_id=0301253f939841e98071973c9b4b0bc5)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:27.117213 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2158dc29-7d39-4ae7-b1ea-af27961ef23a tempest-VolumeQuotaClassesTest-1990960567 tempest-VolumeQuotaClassesTest-1990960567-project-admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:27.120448 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-aa4b75f9-87d9-4797-8b78-4fa1494edd99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fce4031fe294d16ad27f0cde9c8120b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:27.136465 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-aa4b75f9-87d9-4797-8b78-4fa1494edd99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fce4031fe294d16ad27f0cde9c8120b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:27.145652 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-aa4b75f9-87d9-4797-8b78-4fa1494edd99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fce4031fe294d16ad27f0cde9c8120b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:27.152249 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 961/1920] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:17:26 2022] DELETE /identity/v3/users/6fce4031fe294d16ad27f0cde9c8120b => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:27.154269 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4b75f9-87d9-4797-8b78-4fa1494edd99 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:27.154524 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-aa4b75f9-87d9-4797-8b78-4fa1494edd99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:27.174840 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2158dc29-7d39-4ae7-b1ea-af27961ef23a tempest-VolumeQuotaClassesTest-1990960567 tempest-VolumeQuotaClassesTest-1990960567-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0301253f939841e98071973c9b4b0bc5', 'request_id': 'req-2158dc29-7d39-4ae7-b1ea-af27961ef23a'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:27.180631 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2158dc29-7d39-4ae7-b1ea-af27961ef23a tempest-VolumeQuotaClassesTest-1990960567 tempest-VolumeQuotaClassesTest-1990960567-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0301253f939841e98071973c9b4b0bc5', 'request_id': 'req-2158dc29-7d39-4ae7-b1ea-af27961ef23a'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:27.190415 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2158dc29-7d39-4ae7-b1ea-af27961ef23a tempest-VolumeQuotaClassesTest-1990960567 tempest-VolumeQuotaClassesTest-1990960567-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0301253f939841e98071973c9b4b0bc5', 'request_id': 'req-2158dc29-7d39-4ae7-b1ea-af27961ef23a'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:27.210229 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2158dc29-7d39-4ae7-b1ea-af27961ef23a tempest-VolumeQuotaClassesTest-1990960567 tempest-VolumeQuotaClassesTest-1990960567-project-admin] The token cache is being invalidate because project 0301253f939841e98071973c9b4b0bc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:17:27.210583 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2158dc29-7d39-4ae7-b1ea-af27961ef23a tempest-VolumeQuotaClassesTest-1990960567 tempest-VolumeQuotaClassesTest-1990960567-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:27.211614 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2158dc29-7d39-4ae7-b1ea-af27961ef23a tempest-VolumeQuotaClassesTest-1990960567 tempest-VolumeQuotaClassesTest-1990960567-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:27.213543 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:27.214527 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 960/1921] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:17:27 2022] DELETE /identity/v3/projects/0301253f939841e98071973c9b4b0bc5 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:27.216874 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30bf427-c6bf-46eb-bedd-2911ed4ed1e9 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:27.217329 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30bf427-c6bf-46eb-bedd-2911ed4ed1e9 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:27.217773 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30bf427-c6bf-46eb-bedd-2911ed4ed1e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:27.217889 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4b75f9-87d9-4797-8b78-4fa1494edd99 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:27.244404 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-a30bf427-c6bf-46eb-bedd-2911ed4ed1e9 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:27.245656 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a30bf427-c6bf-46eb-bedd-2911ed4ed1e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:27.299876 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2b980d-c5af-467a-99da-6595c39dadb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:27.300844 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b980d-c5af-467a-99da-6595c39dadb0 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:27.301309 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b980d-c5af-467a-99da-6595c39dadb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:27.301738 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b980d-c5af-467a-99da-6595c39dadb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:27.304696 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-6c2b980d-c5af-467a-99da-6595c39dadb0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:27.320157 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 961/1922] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:17:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:27.330829 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30bf427-c6bf-46eb-bedd-2911ed4ed1e9 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:27.332554 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a30bf427-c6bf-46eb-bedd-2911ed4ed1e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:27.346161 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30bf427-c6bf-46eb-bedd-2911ed4ed1e9 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:27.351998 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f086da8-8d1b-4262-b9d7-40bb9d2d9f7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:27.352701 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f086da8-8d1b-4262-b9d7-40bb9d2d9f7d service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:27.353009 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f086da8-8d1b-4262-b9d7-40bb9d2d9f7d service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:27.353423 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f086da8-8d1b-4262-b9d7-40bb9d2d9f7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:27.366348 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f086da8-8d1b-4262-b9d7-40bb9d2d9f7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:27.367843 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f086da8-8d1b-4262-b9d7-40bb9d2d9f7d service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:27.379133 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 962/1923] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:17:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:17:27.389909 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2b980d-c5af-467a-99da-6595c39dadb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:27.391250 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2b980d-c5af-467a-99da-6595c39dadb0 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:27.402103 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 962/1924] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:17:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:17:27.456913 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f086da8-8d1b-4262-b9d7-40bb9d2d9f7d service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:27.458909 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5f086da8-8d1b-4262-b9d7-40bb9d2d9f7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:27.474023 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f086da8-8d1b-4262-b9d7-40bb9d2d9f7d service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:27.480376 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d6dc9a-3f68-414b-a056-5a9499cfb197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:27.481066 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d6dc9a-3f68-414b-a056-5a9499cfb197 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:27.481344 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d6dc9a-3f68-414b-a056-5a9499cfb197 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:27.481610 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d6dc9a-3f68-414b-a056-5a9499cfb197 admin admin] PATH_INFO: `/v3/users/f5f9e500946547f7a00752350f10bbaf` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:27.482849 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d6dc9a-3f68-414b-a056-5a9499cfb197 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5f9e500946547f7a00752350f10bbaf)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:27.484880 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d6dc9a-3f68-414b-a056-5a9499cfb197 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:27.546160 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a2d6dc9a-3f68-414b-a056-5a9499cfb197 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5f9e500946547f7a00752350f10bbaf'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:27.547726 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a2d6dc9a-3f68-414b-a056-5a9499cfb197 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5f9e500946547f7a00752350f10bbaf'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:27.565367 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a2d6dc9a-3f68-414b-a056-5a9499cfb197 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5f9e500946547f7a00752350f10bbaf'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:27.581166 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:27.592353 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a2d6dc9a-3f68-414b-a056-5a9499cfb197 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5f9e500946547f7a00752350f10bbaf'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:27.596038 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a2d6dc9a-3f68-414b-a056-5a9499cfb197 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5f9e500946547f7a00752350f10bbaf'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:27.630539 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 963/1925] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:17:27 2022] DELETE /identity/v3/users/f5f9e500946547f7a00752350f10bbaf => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:27.636890 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc26c1d3-bacb-4ef6-bbac-d427649bd653 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:27.637176 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc26c1d3-bacb-4ef6-bbac-d427649bd653 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:27.637454 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc26c1d3-bacb-4ef6-bbac-d427649bd653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:27.673004 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-dc26c1d3-bacb-4ef6-bbac-d427649bd653 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:27.676534 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-dc26c1d3-bacb-4ef6-bbac-d427649bd653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:27.919155 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 963/1926] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:17:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:27.937830 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d6dc9a-3f68-414b-a056-5a9499cfb197 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:27.939594 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a2d6dc9a-3f68-414b-a056-5a9499cfb197 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:27.992379 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc26c1d3-bacb-4ef6-bbac-d427649bd653 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:27.993969 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-dc26c1d3-bacb-4ef6-bbac-d427649bd653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:28.014167 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc26c1d3-bacb-4ef6-bbac-d427649bd653 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:28.022625 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9bdfac-aa6b-4487-a7b7-2805c099d0cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:28.023389 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9bdfac-aa6b-4487-a7b7-2805c099d0cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:28.023850 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9bdfac-aa6b-4487-a7b7-2805c099d0cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:28.060537 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9bdfac-aa6b-4487-a7b7-2805c099d0cf admin admin] PATH_INFO: `/v3/projects/9693fb70c24a4220b2acf6446aa87a70` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:28.068118 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9bdfac-aa6b-4487-a7b7-2805c099d0cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=9693fb70c24a4220b2acf6446aa87a70)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:28.074506 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9bdfac-aa6b-4487-a7b7-2805c099d0cf admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:28.089171 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d6dc9a-3f68-414b-a056-5a9499cfb197 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:28.099403 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63443463-f6a2-4edb-8c8c-9fdabf2a387a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:28.100985 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63443463-f6a2-4edb-8c8c-9fdabf2a387a service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:28.102357 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63443463-f6a2-4edb-8c8c-9fdabf2a387a service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:28.102851 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63443463-f6a2-4edb-8c8c-9fdabf2a387a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:28.127196 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63443463-f6a2-4edb-8c8c-9fdabf2a387a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:28.131284 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63443463-f6a2-4edb-8c8c-9fdabf2a387a service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:28.154945 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 964/1927] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:17:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:17:28.159868 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63443463-f6a2-4edb-8c8c-9fdabf2a387a service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:28.171727 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-63443463-f6a2-4edb-8c8c-9fdabf2a387a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:28.204298 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63443463-f6a2-4edb-8c8c-9fdabf2a387a service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:28.207994 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-aa9bdfac-aa6b-4487-a7b7-2805c099d0cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9693fb70c24a4220b2acf6446aa87a70', 'request_id': 'req-aa9bdfac-aa6b-4487-a7b7-2805c099d0cf'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:28.211383 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:28.225729 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-aa9bdfac-aa6b-4487-a7b7-2805c099d0cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9693fb70c24a4220b2acf6446aa87a70', 'request_id': 'req-aa9bdfac-aa6b-4487-a7b7-2805c099d0cf'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:28.236978 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-aa9bdfac-aa6b-4487-a7b7-2805c099d0cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9693fb70c24a4220b2acf6446aa87a70', 'request_id': 'req-aa9bdfac-aa6b-4487-a7b7-2805c099d0cf'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:28.256793 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-aa9bdfac-aa6b-4487-a7b7-2805c099d0cf admin admin] The token cache is being invalidate because project 9693fb70c24a4220b2acf6446aa87a70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:17:28.257226 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-aa9bdfac-aa6b-4487-a7b7-2805c099d0cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:28.258035 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-aa9bdfac-aa6b-4487-a7b7-2805c099d0cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:28.260042 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 964/1928] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:17:27 2022] DELETE /identity/v3/projects/9693fb70c24a4220b2acf6446aa87a70 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:17:28.268432 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deda9e0-bae2-424c-a217-300a7a8daca2 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:28.269362 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deda9e0-bae2-424c-a217-300a7a8daca2 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:28.269362 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deda9e0-bae2-424c-a217-300a7a8daca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:28.306371 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-5deda9e0-bae2-424c-a217-300a7a8daca2 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:28.308176 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5deda9e0-bae2-424c-a217-300a7a8daca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:28.321905 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50c8c4e-10a8-44d1-ba92-c1683719740e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:28.322841 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50c8c4e-10a8-44d1-ba92-c1683719740e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:28.323450 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50c8c4e-10a8-44d1-ba92-c1683719740e admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:28.323900 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50c8c4e-10a8-44d1-ba92-c1683719740e admin admin] PATH_INFO: `/v3/projects/e3abc7d1ffc140cea2fd3e4afc677285` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:28.325743 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50c8c4e-10a8-44d1-ba92-c1683719740e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3abc7d1ffc140cea2fd3e4afc677285)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:28.328991 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50c8c4e-10a8-44d1-ba92-c1683719740e admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:28.405344 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e50c8c4e-10a8-44d1-ba92-c1683719740e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3abc7d1ffc140cea2fd3e4afc677285', 'request_id': 'req-e50c8c4e-10a8-44d1-ba92-c1683719740e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:28.409327 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 965/1929] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:17:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:17:28.415949 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d524d8d-d8b1-4bc8-a6e7-31823b9c6fa7 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:28.416365 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d524d8d-d8b1-4bc8-a6e7-31823b9c6fa7 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:28.416857 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d524d8d-d8b1-4bc8-a6e7-31823b9c6fa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:28.419185 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e50c8c4e-10a8-44d1-ba92-c1683719740e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3abc7d1ffc140cea2fd3e4afc677285', 'request_id': 'req-e50c8c4e-10a8-44d1-ba92-c1683719740e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:28.443405 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e50c8c4e-10a8-44d1-ba92-c1683719740e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3abc7d1ffc140cea2fd3e4afc677285', 'request_id': 'req-e50c8c4e-10a8-44d1-ba92-c1683719740e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:28.468838 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e50c8c4e-10a8-44d1-ba92-c1683719740e admin admin] The token cache is being invalidate because project e3abc7d1ffc140cea2fd3e4afc677285 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:17:28.469449 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e50c8c4e-10a8-44d1-ba92-c1683719740e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:28.470715 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e50c8c4e-10a8-44d1-ba92-c1683719740e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:28.473683 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 965/1930] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:17:28 2022] DELETE /identity/v3/projects/e3abc7d1ffc140cea2fd3e4afc677285 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:28.474440 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-8d524d8d-d8b1-4bc8-a6e7-31823b9c6fa7 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:28.476315 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8d524d8d-d8b1-4bc8-a6e7-31823b9c6fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:28.479146 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50c8c4e-10a8-44d1-ba92-c1683719740e admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:28.481441 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e50c8c4e-10a8-44d1-ba92-c1683719740e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:28.533528 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:28.570329 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 966/1931] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:17:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:28.574907 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d524d8d-d8b1-4bc8-a6e7-31823b9c6fa7 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:28.590724 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d524d8d-d8b1-4bc8-a6e7-31823b9c6fa7 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:28.597363 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24768e97-10eb-4478-8b2c-ed874c3d3f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:28.598159 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24768e97-10eb-4478-8b2c-ed874c3d3f41 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:28.598494 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24768e97-10eb-4478-8b2c-ed874c3d3f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:28.598839 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24768e97-10eb-4478-8b2c-ed874c3d3f41 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:28.599559 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24768e97-10eb-4478-8b2c-ed874c3d3f41 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:17:28.599933 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24768e97-10eb-4478-8b2c-ed874c3d3f41 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:28.602068 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24768e97-10eb-4478-8b2c-ed874c3d3f41 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:28.612430 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50c8c4e-10a8-44d1-ba92-c1683719740e admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:28.618045 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9537db8c-acf8-4487-917b-e5a76103b90f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:28.618692 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9537db8c-acf8-4487-917b-e5a76103b90f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:28.618965 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9537db8c-acf8-4487-917b-e5a76103b90f admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:28.619236 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9537db8c-acf8-4487-917b-e5a76103b90f admin admin] PATH_INFO: `/v3/users/8e193a34ba9043baba0091c70e03659f` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:28.620482 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9537db8c-acf8-4487-917b-e5a76103b90f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e193a34ba9043baba0091c70e03659f)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:28.622776 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9537db8c-acf8-4487-917b-e5a76103b90f admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:28.623935 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 967/1932] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 6 16:17:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:17:28.625008 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24768e97-10eb-4478-8b2c-ed874c3d3f41 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:28.636532 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24768e97-10eb-4478-8b2c-ed874c3d3f41 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:28.642531 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be359f1c-4d15-43b7-8245-a3b63d205abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:28.643150 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be359f1c-4d15-43b7-8245-a3b63d205abe admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:28.643535 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be359f1c-4d15-43b7-8245-a3b63d205abe admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:28.643839 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be359f1c-4d15-43b7-8245-a3b63d205abe admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:28.644508 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be359f1c-4d15-43b7-8245-a3b63d205abe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:28.646532 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be359f1c-4d15-43b7-8245-a3b63d205abe admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:28.685232 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 968/1933] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:17:28 2022] POST /identity/v3/projects => generated 373 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:17:28.686173 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be359f1c-4d15-43b7-8245-a3b63d205abe admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:28.689394 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-be359f1c-4d15-43b7-8245-a3b63d205abe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:28.709678 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9537db8c-acf8-4487-917b-e5a76103b90f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e193a34ba9043baba0091c70e03659f'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:28.710778 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9537db8c-acf8-4487-917b-e5a76103b90f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e193a34ba9043baba0091c70e03659f'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:28.724843 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9537db8c-acf8-4487-917b-e5a76103b90f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e193a34ba9043baba0091c70e03659f'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:28.738536 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9537db8c-acf8-4487-917b-e5a76103b90f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e193a34ba9043baba0091c70e03659f'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:28.739318 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9537db8c-acf8-4487-917b-e5a76103b90f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e193a34ba9043baba0091c70e03659f'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:28.750072 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:INSERT=2,keystone:DELETE=3,keystone:SELECT=95 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:28.759124 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 966/1934] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:17:28 2022] DELETE /identity/v3/users/8e193a34ba9043baba0091c70e03659f => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:28.761765 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9537db8c-acf8-4487-917b-e5a76103b90f admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:28.770083 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-9537db8c-acf8-4487-917b-e5a76103b90f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:28.778507 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be359f1c-4d15-43b7-8245-a3b63d205abe admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:28.887014 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cd6449-db07-434c-b52a-d384f232f257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:28.888083 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd6449-db07-434c-b52a-d384f232f257 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:28.888633 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd6449-db07-434c-b52a-d384f232f257 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:28.889401 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd6449-db07-434c-b52a-d384f232f257 admin admin] PATH_INFO: `/v3/projects/c6e56a2ac043457b81a0c3d01f8dab2c` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:28.892831 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cd6449-db07-434c-b52a-d384f232f257 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6e56a2ac043457b81a0c3d01f8dab2c)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:28.896220 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cd6449-db07-434c-b52a-d384f232f257 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:28.909357 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9537db8c-acf8-4487-917b-e5a76103b90f admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:28.942257 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b963199d-59d5-47b0-b128-46a9d268c22e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:28.945023 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b963199d-59d5-47b0-b128-46a9d268c22e admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:28.945023 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b963199d-59d5-47b0-b128-46a9d268c22e admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:28.945337 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b963199d-59d5-47b0-b128-46a9d268c22e admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:28.946288 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b963199d-59d5-47b0-b128-46a9d268c22e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:28.951674 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b963199d-59d5-47b0-b128-46a9d268c22e admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:28.996703 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 967/1935] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:17:28 2022] POST /identity/v3/users => generated 287 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:17:28.999243 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b963199d-59d5-47b0-b128-46a9d268c22e admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:29.002041 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-f9cd6449-db07-434c-b52a-d384f232f257 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6e56a2ac043457b81a0c3d01f8dab2c', 'request_id': 'req-f9cd6449-db07-434c-b52a-d384f232f257'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:29.014329 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-f9cd6449-db07-434c-b52a-d384f232f257 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6e56a2ac043457b81a0c3d01f8dab2c', 'request_id': 'req-f9cd6449-db07-434c-b52a-d384f232f257'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:29.015352 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b963199d-59d5-47b0-b128-46a9d268c22e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:29.024671 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-f9cd6449-db07-434c-b52a-d384f232f257 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6e56a2ac043457b81a0c3d01f8dab2c', 'request_id': 'req-f9cd6449-db07-434c-b52a-d384f232f257'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:29.043405 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b963199d-59d5-47b0-b128-46a9d268c22e admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:29.046628 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-f9cd6449-db07-434c-b52a-d384f232f257 admin admin] The token cache is being invalidate because project c6e56a2ac043457b81a0c3d01f8dab2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:17:29.047036 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-f9cd6449-db07-434c-b52a-d384f232f257 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:29.047812 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-f9cd6449-db07-434c-b52a-d384f232f257 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:29.050022 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 969/1936] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:17:28 2022] DELETE /identity/v3/projects/c6e56a2ac043457b81a0c3d01f8dab2c => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:29.051766 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cd6449-db07-434c-b52a-d384f232f257 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:29.053945 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f9cd6449-db07-434c-b52a-d384f232f257 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:29.083013 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:29.144208 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cd6449-db07-434c-b52a-d384f232f257 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:29.149076 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3b8d5e-8ff4-483a-a8d9-6298daaaf9b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:29.150155 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3b8d5e-8ff4-483a-a8d9-6298daaaf9b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:29.150838 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3b8d5e-8ff4-483a-a8d9-6298daaaf9b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:29.151331 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3b8d5e-8ff4-483a-a8d9-6298daaaf9b7 admin admin] PATH_INFO: `/v3/users/1a5951a7771c42e4bbae1aa56bded0bf` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:29.152922 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afbcf72-879b-4f51-9eda-d30148452add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:29.153519 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3b8d5e-8ff4-483a-a8d9-6298daaaf9b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a5951a7771c42e4bbae1aa56bded0bf)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:29.154159 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afbcf72-879b-4f51-9eda-d30148452add admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:29.154655 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afbcf72-879b-4f51-9eda-d30148452add admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:29.155126 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afbcf72-879b-4f51-9eda-d30148452add admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:29.155984 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afbcf72-879b-4f51-9eda-d30148452add admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:29.157271 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3b8d5e-8ff4-483a-a8d9-6298daaaf9b7 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:29.157962 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afbcf72-879b-4f51-9eda-d30148452add admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:29.170111 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 970/1937] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:17:29 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:17:29.171102 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afbcf72-879b-4f51-9eda-d30148452add admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:29.173268 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8afbcf72-879b-4f51-9eda-d30148452add admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:29.188827 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afbcf72-879b-4f51-9eda-d30148452add admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:29.197336 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46ae6ee-dd0d-49d4-aadb-1449bde176af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:29.201520 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46ae6ee-dd0d-49d4-aadb-1449bde176af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:29.202018 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46ae6ee-dd0d-49d4-aadb-1449bde176af admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:29.202448 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46ae6ee-dd0d-49d4-aadb-1449bde176af admin admin] PATH_INFO: `/v3/users/7e5420775faa44bc88a817caee3d5a4c` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:29.205752 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46ae6ee-dd0d-49d4-aadb-1449bde176af admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e5420775faa44bc88a817caee3d5a4c)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:29.207863 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46ae6ee-dd0d-49d4-aadb-1449bde176af admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:29.233211 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:29.259228 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-de3b8d5e-8ff4-483a-a8d9-6298daaaf9b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a5951a7771c42e4bbae1aa56bded0bf'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:29.260948 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-de3b8d5e-8ff4-483a-a8d9-6298daaaf9b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a5951a7771c42e4bbae1aa56bded0bf'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:29.276551 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-de3b8d5e-8ff4-483a-a8d9-6298daaaf9b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a5951a7771c42e4bbae1aa56bded0bf'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:29.300941 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-de3b8d5e-8ff4-483a-a8d9-6298daaaf9b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a5951a7771c42e4bbae1aa56bded0bf'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:29.302352 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-de3b8d5e-8ff4-483a-a8d9-6298daaaf9b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a5951a7771c42e4bbae1aa56bded0bf'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:29.305841 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b46ae6ee-dd0d-49d4-aadb-1449bde176af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e5420775faa44bc88a817caee3d5a4c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:29.307040 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b46ae6ee-dd0d-49d4-aadb-1449bde176af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e5420775faa44bc88a817caee3d5a4c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:29.318345 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b46ae6ee-dd0d-49d4-aadb-1449bde176af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e5420775faa44bc88a817caee3d5a4c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:29.321005 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 968/1938] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:17:28 2022] DELETE /identity/v3/users/1a5951a7771c42e4bbae1aa56bded0bf => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:29.322570 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3b8d5e-8ff4-483a-a8d9-6298daaaf9b7 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:29.324250 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-de3b8d5e-8ff4-483a-a8d9-6298daaaf9b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:29.335865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b46ae6ee-dd0d-49d4-aadb-1449bde176af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e5420775faa44bc88a817caee3d5a4c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:29.349600 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b46ae6ee-dd0d-49d4-aadb-1449bde176af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e5420775faa44bc88a817caee3d5a4c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:29.356790 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 971/1939] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:17:29 2022] DELETE /identity/v3/users/7e5420775faa44bc88a817caee3d5a4c => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:29.357503 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46ae6ee-dd0d-49d4-aadb-1449bde176af admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:29.358849 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b46ae6ee-dd0d-49d4-aadb-1449bde176af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:29.428775 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3b8d5e-8ff4-483a-a8d9-6298daaaf9b7 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:29.433474 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46ae6ee-dd0d-49d4-aadb-1449bde176af admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:29.435890 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47745c35-5290-481c-920c-7f8f46d21465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:29.437347 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47745c35-5290-481c-920c-7f8f46d21465 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:29.437939 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47745c35-5290-481c-920c-7f8f46d21465 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:29.438209 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47745c35-5290-481c-920c-7f8f46d21465 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:29.439339 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7486c84-8d02-44f9-99f5-d128e1c96f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:29.440387 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7486c84-8d02-44f9-99f5-d128e1c96f43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:29.440736 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7486c84-8d02-44f9-99f5-d128e1c96f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:29.441119 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7486c84-8d02-44f9-99f5-d128e1c96f43 admin admin] PATH_INFO: `/v3/projects/d9f207d474314ca4a8c8083323c3a7ca` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:29.442397 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7486c84-8d02-44f9-99f5-d128e1c96f43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9f207d474314ca4a8c8083323c3a7ca)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:29.444424 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7486c84-8d02-44f9-99f5-d128e1c96f43 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:29.445802 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47745c35-5290-481c-920c-7f8f46d21465 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:29.448998 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47745c35-5290-481c-920c-7f8f46d21465 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:29.466199 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 969/1940] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:17:29 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:17:29.468996 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422dc7a-3ae6-463c-88a1-8d26ea3780d0 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:29.469276 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422dc7a-3ae6-463c-88a1-8d26ea3780d0 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:29.469546 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422dc7a-3ae6-463c-88a1-8d26ea3780d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:29.495226 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-d422dc7a-3ae6-463c-88a1-8d26ea3780d0 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:29.496477 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d422dc7a-3ae6-463c-88a1-8d26ea3780d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:29.503066 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d7486c84-8d02-44f9-99f5-d128e1c96f43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9f207d474314ca4a8c8083323c3a7ca', 'request_id': 'req-d7486c84-8d02-44f9-99f5-d128e1c96f43'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:29.530221 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d7486c84-8d02-44f9-99f5-d128e1c96f43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9f207d474314ca4a8c8083323c3a7ca', 'request_id': 'req-d7486c84-8d02-44f9-99f5-d128e1c96f43'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:29.539391 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d7486c84-8d02-44f9-99f5-d128e1c96f43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9f207d474314ca4a8c8083323c3a7ca', 'request_id': 'req-d7486c84-8d02-44f9-99f5-d128e1c96f43'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:29.560995 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d7486c84-8d02-44f9-99f5-d128e1c96f43 admin admin] The token cache is being invalidate because project d9f207d474314ca4a8c8083323c3a7ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:17:29.561226 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d7486c84-8d02-44f9-99f5-d128e1c96f43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:29.562201 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d7486c84-8d02-44f9-99f5-d128e1c96f43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:29.564463 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 972/1941] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:17:29 2022] DELETE /identity/v3/projects/d9f207d474314ca4a8c8083323c3a7ca => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:29.565306 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7486c84-8d02-44f9-99f5-d128e1c96f43 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:29.566693 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d7486c84-8d02-44f9-99f5-d128e1c96f43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:29.568211 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:29.597055 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:29.608279 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 970/1942] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:17:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:29.615317 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc01d5f2-f79b-4137-9ea4-c26494d053d2 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:29.615680 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc01d5f2-f79b-4137-9ea4-c26494d053d2 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:29.616021 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc01d5f2-f79b-4137-9ea4-c26494d053d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:29.647179 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-fc01d5f2-f79b-4137-9ea4-c26494d053d2 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:29.649916 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-fc01d5f2-f79b-4137-9ea4-c26494d053d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:29.668975 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7486c84-8d02-44f9-99f5-d128e1c96f43 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:29.669199 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 971/1943] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:17:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:29.669199 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc01d5f2-f79b-4137-9ea4-c26494d053d2 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:29.669347 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d7d1fc-8b24-4cfa-b1a6-b5742639181a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:29.669347 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d7d1fc-8b24-4cfa-b1a6-b5742639181a admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:29.669347 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d7d1fc-8b24-4cfa-b1a6-b5742639181a admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:29.669347 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d7d1fc-8b24-4cfa-b1a6-b5742639181a admin admin] PATH_INFO: `/v3/projects/61cede192fdd4fbca3e14fc7c17a9ccd/users/2a575e0f779a485e9918db75f1c7da7f/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:29.671543 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-fc01d5f2-f79b-4137-9ea4-c26494d053d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:29.709607 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d7d1fc-8b24-4cfa-b1a6-b5742639181a admin admin] RBAC: Authorizing `identity:create_grant(project_id=61cede192fdd4fbca3e14fc7c17a9ccd, user_id=2a575e0f779a485e9918db75f1c7da7f, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:29.723987 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d7d1fc-8b24-4cfa-b1a6-b5742639181a admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:29.756050 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 973/1944] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:17:29 2022] PUT /identity/v3/projects/61cede192fdd4fbca3e14fc7c17a9ccd/users/2a575e0f779a485e9918db75f1c7da7f/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:29.757367 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5306c26-db77-496b-9787-5eae31cae1dd None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:29.757655 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5306c26-db77-496b-9787-5eae31cae1dd None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:29.757933 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5306c26-db77-496b-9787-5eae31cae1dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:29.782744 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc01d5f2-f79b-4137-9ea4-c26494d053d2 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:29.793992 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-d5306c26-db77-496b-9787-5eae31cae1dd None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:29.796571 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d5306c26-db77-496b-9787-5eae31cae1dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:29.901994 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84441482-3240-4cec-8dab-b447ec60ab55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:29.904212 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84441482-3240-4cec-8dab-b447ec60ab55 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:29.904746 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84441482-3240-4cec-8dab-b447ec60ab55 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:29.905630 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84441482-3240-4cec-8dab-b447ec60ab55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:29.914585 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 974/1945] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:17:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:29.919346 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f38712-1d1c-4886-a83b-43b0a9239408 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:29.919634 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f38712-1d1c-4886-a83b-43b0a9239408 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:29.919906 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f38712-1d1c-4886-a83b-43b0a9239408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:29.929078 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84441482-3240-4cec-8dab-b447ec60ab55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:29.930557 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84441482-3240-4cec-8dab-b447ec60ab55 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:29.949622 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 972/1946] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:17:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:17:29.966610 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-f6f38712-1d1c-4886-a83b-43b0a9239408 None None] MFA Rules not processed for user `2a575e0f779a485e9918db75f1c7da7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:29.967854 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f6f38712-1d1c-4886-a83b-43b0a9239408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:29.970777 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84441482-3240-4cec-8dab-b447ec60ab55 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:29.973545 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-84441482-3240-4cec-8dab-b447ec60ab55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:29.999510 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84441482-3240-4cec-8dab-b447ec60ab55 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:30.006529 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc97d701-d3bd-42a1-b728-17cb5f540ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:30.007536 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc97d701-d3bd-42a1-b728-17cb5f540ef7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:30.008025 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc97d701-d3bd-42a1-b728-17cb5f540ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:30.008656 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc97d701-d3bd-42a1-b728-17cb5f540ef7 admin admin] PATH_INFO: `/v3/projects/b6dea956937c4e6d9955c0499d4ff069` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:30.010832 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc97d701-d3bd-42a1-b728-17cb5f540ef7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6dea956937c4e6d9955c0499d4ff069)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:30.013665 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc97d701-d3bd-42a1-b728-17cb5f540ef7 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:30.107050 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 975/1947] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:17:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:30.108230 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f38712-1d1c-4886-a83b-43b0a9239408 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:30.139422 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f38712-1d1c-4886-a83b-43b0a9239408 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:30.170245 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-bc97d701-d3bd-42a1-b728-17cb5f540ef7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6dea956937c4e6d9955c0499d4ff069', 'request_id': 'req-bc97d701-d3bd-42a1-b728-17cb5f540ef7'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:30.190477 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-bc97d701-d3bd-42a1-b728-17cb5f540ef7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6dea956937c4e6d9955c0499d4ff069', 'request_id': 'req-bc97d701-d3bd-42a1-b728-17cb5f540ef7'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:30.205675 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-bc97d701-d3bd-42a1-b728-17cb5f540ef7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6dea956937c4e6d9955c0499d4ff069', 'request_id': 'req-bc97d701-d3bd-42a1-b728-17cb5f540ef7'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:30.212033 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:30.231997 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-bc97d701-d3bd-42a1-b728-17cb5f540ef7 admin admin] The token cache is being invalidate because project b6dea956937c4e6d9955c0499d4ff069 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:17:30.231997 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-bc97d701-d3bd-42a1-b728-17cb5f540ef7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:30.232361 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-bc97d701-d3bd-42a1-b728-17cb5f540ef7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:30.233964 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 973/1948] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:17:29 2022] DELETE /identity/v3/projects/b6dea956937c4e6d9955c0499d4ff069 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:30.234914 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc97d701-d3bd-42a1-b728-17cb5f540ef7 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:30.236405 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-bc97d701-d3bd-42a1-b728-17cb5f540ef7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:30.283197 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:30.305285 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f637e3aa-bf57-4ca3-95cd-3039d4d359c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:30.305673 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f637e3aa-bf57-4ca3-95cd-3039d4d359c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:30.306632 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f637e3aa-bf57-4ca3-95cd-3039d4d359c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:30.306632 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f637e3aa-bf57-4ca3-95cd-3039d4d359c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:30.307062 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f637e3aa-bf57-4ca3-95cd-3039d4d359c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:17:30.307399 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f637e3aa-bf57-4ca3-95cd-3039d4d359c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:30.312734 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f637e3aa-bf57-4ca3-95cd-3039d4d359c4 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:30.344479 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 976/1949] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 6 16:17:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:17:30.344479 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c33fdfb-d38c-4808-beb6-109fed3b7255 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:30.344479 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c33fdfb-d38c-4808-beb6-109fed3b7255 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:30.344479 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c33fdfb-d38c-4808-beb6-109fed3b7255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:30.361940 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc97d701-d3bd-42a1-b728-17cb5f540ef7 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:30.373362 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef1aca8-a700-4d74-a6e9-0c28e4e21a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:30.374867 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef1aca8-a700-4d74-a6e9-0c28e4e21a58 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:30.375423 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef1aca8-a700-4d74-a6e9-0c28e4e21a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:30.376004 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef1aca8-a700-4d74-a6e9-0c28e4e21a58 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:30.378096 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef1aca8-a700-4d74-a6e9-0c28e4e21a58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:30.382390 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef1aca8-a700-4d74-a6e9-0c28e4e21a58 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:30.384564 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-1c33fdfb-d38c-4808-beb6-109fed3b7255 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:30.386741 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-1c33fdfb-d38c-4808-beb6-109fed3b7255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:30.397413 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 977/1950] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:17:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:30.398365 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c33fdfb-d38c-4808-beb6-109fed3b7255 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:30.400033 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-1c33fdfb-d38c-4808-beb6-109fed3b7255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:30.417171 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c33fdfb-d38c-4808-beb6-109fed3b7255 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:30.429785 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd9505f-de1d-4f0e-ac02-656b3c1ec523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:30.430842 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd9505f-de1d-4f0e-ac02-656b3c1ec523 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:30.431298 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd9505f-de1d-4f0e-ac02-656b3c1ec523 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:30.431752 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd9505f-de1d-4f0e-ac02-656b3c1ec523 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:30.433342 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd9505f-de1d-4f0e-ac02-656b3c1ec523 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:30.435561 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd9505f-de1d-4f0e-ac02-656b3c1ec523 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:30.443282 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 974/1951] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:17:30 2022] POST /identity/v3/projects => generated 375 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:17:30.445068 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0457c8a5-52bd-451f-a9b9-af9b9bdce57e None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:30.445307 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0457c8a5-52bd-451f-a9b9-af9b9bdce57e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:30.445632 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0457c8a5-52bd-451f-a9b9-af9b9bdce57e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:30.469373 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-0457c8a5-52bd-451f-a9b9-af9b9bdce57e None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:30.471406 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 978/1952] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:17:30 2022] POST /identity/v3/projects => generated 375 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:17:30.471534 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd9505f-de1d-4f0e-ac02-656b3c1ec523 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:30.473215 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0457c8a5-52bd-451f-a9b9-af9b9bdce57e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:30.476429 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-2bd9505f-de1d-4f0e-ac02-656b3c1ec523 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:30.549512 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 975/1953] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:17:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:30.550400 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0457c8a5-52bd-451f-a9b9-af9b9bdce57e None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:30.556400 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd9505f-de1d-4f0e-ac02-656b3c1ec523 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:30.562531 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dfc9a1-8b7c-49dd-88d8-3a5e1accb7c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:30.563262 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dfc9a1-8b7c-49dd-88d8-3a5e1accb7c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:30.563558 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dfc9a1-8b7c-49dd-88d8-3a5e1accb7c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:30.563665 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0457c8a5-52bd-451f-a9b9-af9b9bdce57e None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:30.563850 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dfc9a1-8b7c-49dd-88d8-3a5e1accb7c6 admin admin] PATH_INFO: `/v3/projects/e5df251fb443471694cba6a295b6c337` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:30.565121 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dfc9a1-8b7c-49dd-88d8-3a5e1accb7c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5df251fb443471694cba6a295b6c337)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:30.567126 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dfc9a1-8b7c-49dd-88d8-3a5e1accb7c6 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:30.570519 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17306b67-585c-4e9c-8ae2-e2ae11f77bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:30.571145 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17306b67-585c-4e9c-8ae2-e2ae11f77bd4 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:30.571425 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17306b67-585c-4e9c-8ae2-e2ae11f77bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:30.571703 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17306b67-585c-4e9c-8ae2-e2ae11f77bd4 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:30.572347 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17306b67-585c-4e9c-8ae2-e2ae11f77bd4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:30.574925 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17306b67-585c-4e9c-8ae2-e2ae11f77bd4 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:30.612917 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 976/1954] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:17:30 2022] POST /identity/v3/users => generated 288 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:17:30.614135 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17306b67-585c-4e9c-8ae2-e2ae11f77bd4 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:30.628604 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17306b67-585c-4e9c-8ae2-e2ae11f77bd4 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:30.634670 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e2dfc9a1-8b7c-49dd-88d8-3a5e1accb7c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5df251fb443471694cba6a295b6c337', 'request_id': 'req-e2dfc9a1-8b7c-49dd-88d8-3a5e1accb7c6'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:30.645768 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e2dfc9a1-8b7c-49dd-88d8-3a5e1accb7c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5df251fb443471694cba6a295b6c337', 'request_id': 'req-e2dfc9a1-8b7c-49dd-88d8-3a5e1accb7c6'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:30.653873 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e2dfc9a1-8b7c-49dd-88d8-3a5e1accb7c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5df251fb443471694cba6a295b6c337', 'request_id': 'req-e2dfc9a1-8b7c-49dd-88d8-3a5e1accb7c6'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:30.667179 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e2dfc9a1-8b7c-49dd-88d8-3a5e1accb7c6 admin admin] The token cache is being invalidate because project e5df251fb443471694cba6a295b6c337 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:17:30.667486 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e2dfc9a1-8b7c-49dd-88d8-3a5e1accb7c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:30.668206 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e2dfc9a1-8b7c-49dd-88d8-3a5e1accb7c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:30.670825 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 979/1955] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:17:30 2022] DELETE /identity/v3/projects/e5df251fb443471694cba6a295b6c337 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:30.671053 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dfc9a1-8b7c-49dd-88d8-3a5e1accb7c6 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:30.672459 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e2dfc9a1-8b7c-49dd-88d8-3a5e1accb7c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:30.704800 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea9fe68-86dd-4359-8aaa-5cd3d3b52160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:30.705493 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea9fe68-86dd-4359-8aaa-5cd3d3b52160 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:30.705783 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea9fe68-86dd-4359-8aaa-5cd3d3b52160 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:30.706057 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea9fe68-86dd-4359-8aaa-5cd3d3b52160 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:30.706898 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea9fe68-86dd-4359-8aaa-5cd3d3b52160 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:30.707754 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:30.709946 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea9fe68-86dd-4359-8aaa-5cd3d3b52160 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:30.740410 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 977/1956] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:17:30 2022] POST /identity/v3/users => generated 288 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:17:30.742109 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea9fe68-86dd-4359-8aaa-5cd3d3b52160 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:30.744129 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0ea9fe68-86dd-4359-8aaa-5cd3d3b52160 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:30.756229 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dfc9a1-8b7c-49dd-88d8-3a5e1accb7c6 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:30.759434 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea9fe68-86dd-4359-8aaa-5cd3d3b52160 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:30.761909 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7019fe18-6825-4eca-83f0-507f63819f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:30.762528 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7019fe18-6825-4eca-83f0-507f63819f99 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:30.762813 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7019fe18-6825-4eca-83f0-507f63819f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:30.763095 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7019fe18-6825-4eca-83f0-507f63819f99 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:30.763756 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7019fe18-6825-4eca-83f0-507f63819f99 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:17:30.764063 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7019fe18-6825-4eca-83f0-507f63819f99 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:30.764518 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0df7b3-e2e2-4fbf-94cc-819ac7d879ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:30.765230 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0df7b3-e2e2-4fbf-94cc-819ac7d879ac admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:30.765518 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0df7b3-e2e2-4fbf-94cc-819ac7d879ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:30.765788 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0df7b3-e2e2-4fbf-94cc-819ac7d879ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:30.766244 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7019fe18-6825-4eca-83f0-507f63819f99 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:30.766448 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0df7b3-e2e2-4fbf-94cc-819ac7d879ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:30.768707 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0df7b3-e2e2-4fbf-94cc-819ac7d879ac admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:30.784469 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 980/1957] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 6 16:17:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:17:30.784469 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b3f69a-06d4-4a87-ad82-e4cb7912e740 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:30.784469 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b3f69a-06d4-4a87-ad82-e4cb7912e740 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:30.784469 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b3f69a-06d4-4a87-ad82-e4cb7912e740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:30.791808 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 978/1958] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:17:30 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:17:30.795524 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0df7b3-e2e2-4fbf-94cc-819ac7d879ac admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:30.797328 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5f0df7b3-e2e2-4fbf-94cc-819ac7d879ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:30.814326 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-d6b3f69a-06d4-4a87-ad82-e4cb7912e740 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:30.818259 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d6b3f69a-06d4-4a87-ad82-e4cb7912e740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:30.818454 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0df7b3-e2e2-4fbf-94cc-819ac7d879ac admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:30.823709 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a2511b-de33-4820-86a3-64b828b3c11d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:30.824504 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a2511b-de33-4820-86a3-64b828b3c11d admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:30.824877 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a2511b-de33-4820-86a3-64b828b3c11d admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:30.825158 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a2511b-de33-4820-86a3-64b828b3c11d admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:30.826208 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a2511b-de33-4820-86a3-64b828b3c11d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:30.827897 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a2511b-de33-4820-86a3-64b828b3c11d admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:30.833022 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 981/1959] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:17:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:30.835085 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b3f69a-06d4-4a87-ad82-e4cb7912e740 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:30.835717 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:30.853361 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 979/1960] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:17:30 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:17:30.857062 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b3f69a-06d4-4a87-ad82-e4cb7912e740 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:30.857977 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a2511b-de33-4820-86a3-64b828b3c11d admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:30.863014 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fee691-e524-48ac-92f0-fde4410ea7c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:30.864143 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fee691-e524-48ac-92f0-fde4410ea7c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:30.864424 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fee691-e524-48ac-92f0-fde4410ea7c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:30.864697 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fee691-e524-48ac-92f0-fde4410ea7c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:30.865536 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fee691-e524-48ac-92f0-fde4410ea7c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:30.867500 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fee691-e524-48ac-92f0-fde4410ea7c3 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:30.879344 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a2511b-de33-4820-86a3-64b828b3c11d admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:30.885774 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57530eee-b045-461e-a63f-2cc56bd78520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:30.887494 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57530eee-b045-461e-a63f-2cc56bd78520 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:30.887969 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57530eee-b045-461e-a63f-2cc56bd78520 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:30.888581 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57530eee-b045-461e-a63f-2cc56bd78520 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:30.889404 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57530eee-b045-461e-a63f-2cc56bd78520 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:30.891368 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57530eee-b045-461e-a63f-2cc56bd78520 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:30.904057 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 980/1961] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:17:30 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:17:30.905568 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba67244-3c43-4a8c-87f1-2cdb33d2a366 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:30.905945 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba67244-3c43-4a8c-87f1-2cdb33d2a366 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:30.906225 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba67244-3c43-4a8c-87f1-2cdb33d2a366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:30.908056 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 982/1962] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:17:30 2022] POST /identity/v3/projects => generated 401 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:17:30.909403 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fee691-e524-48ac-92f0-fde4410ea7c3 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:30.929235 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-4ba67244-3c43-4a8c-87f1-2cdb33d2a366 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:30.930427 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-4ba67244-3c43-4a8c-87f1-2cdb33d2a366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:30.990235 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fee691-e524-48ac-92f0-fde4410ea7c3 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:30.997434 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41640422-31d2-4f2c-92a9-75994be9f6ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:30.998051 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41640422-31d2-4f2c-92a9-75994be9f6ab admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:30.998328 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41640422-31d2-4f2c-92a9-75994be9f6ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:30.998605 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41640422-31d2-4f2c-92a9-75994be9f6ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:30.999266 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41640422-31d2-4f2c-92a9-75994be9f6ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:31.000948 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41640422-31d2-4f2c-92a9-75994be9f6ab admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:31.004688 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 981/1963] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:17:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:31.005719 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba67244-3c43-4a8c-87f1-2cdb33d2a366 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:31.014195 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 983/1964] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:17:30 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:17:31.018015 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba67244-3c43-4a8c-87f1-2cdb33d2a366 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:31.018162 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41640422-31d2-4f2c-92a9-75994be9f6ab admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:31.024840 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e23af43-30df-4594-965e-49160f93f6b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:31.025737 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e23af43-30df-4594-965e-49160f93f6b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:31.026096 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e23af43-30df-4594-965e-49160f93f6b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:31.026373 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e23af43-30df-4594-965e-49160f93f6b8 admin admin] PATH_INFO: `/v3/projects/510f6230c5d545338f64da15c5f2ea28/users/8954a22e9f324f52b432801f19215e1d/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:31.028063 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41640422-31d2-4f2c-92a9-75994be9f6ab admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:31.033365 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac21145-3c25-462e-b156-45809b80d43b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:31.033976 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac21145-3c25-462e-b156-45809b80d43b admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:31.034255 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac21145-3c25-462e-b156-45809b80d43b admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:31.034558 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac21145-3c25-462e-b156-45809b80d43b admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:31.035312 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac21145-3c25-462e-b156-45809b80d43b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:31.037382 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac21145-3c25-462e-b156-45809b80d43b admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:31.043667 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e23af43-30df-4594-965e-49160f93f6b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=510f6230c5d545338f64da15c5f2ea28, user_id=8954a22e9f324f52b432801f19215e1d, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:31.046552 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e23af43-30df-4594-965e-49160f93f6b8 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:31.065337 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 982/1965] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:17:31 2022] PUT /identity/v3/projects/510f6230c5d545338f64da15c5f2ea28/users/8954a22e9f324f52b432801f19215e1d/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:31.067408 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e23af43-30df-4594-965e-49160f93f6b8 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:31.069751 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 984/1966] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:17:31 2022] POST /identity/v3/users => generated 301 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:17:31.071329 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac21145-3c25-462e-b156-45809b80d43b admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:31.087453 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac21145-3c25-462e-b156-45809b80d43b admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:31.180667 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e23af43-30df-4594-965e-49160f93f6b8 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:31.182269 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5a55a1-1a0a-495c-be22-27689028464e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:31.183464 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5a55a1-1a0a-495c-be22-27689028464e admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:31.184173 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5a55a1-1a0a-495c-be22-27689028464e admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:31.184593 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5a55a1-1a0a-495c-be22-27689028464e admin admin] PATH_INFO: `/v3/projects/53173c7ff08343adace7682eb5997f94/users/2a5bec8038c64151af84cb1c9d5cfd35/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:31.186205 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fec22f-5b46-44d4-a217-694cf73fe418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:31.187585 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fec22f-5b46-44d4-a217-694cf73fe418 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:31.187869 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fec22f-5b46-44d4-a217-694cf73fe418 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:31.188139 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fec22f-5b46-44d4-a217-694cf73fe418 admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:31.188928 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fec22f-5b46-44d4-a217-694cf73fe418 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:17:31.189675 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fec22f-5b46-44d4-a217-694cf73fe418 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:31.191388 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fec22f-5b46-44d4-a217-694cf73fe418 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:31.204600 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5a55a1-1a0a-495c-be22-27689028464e admin admin] RBAC: Authorizing `identity:create_grant(project_id=53173c7ff08343adace7682eb5997f94, user_id=2a5bec8038c64151af84cb1c9d5cfd35, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:31.205821 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 983/1967] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 6 16:17:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:17:31.207260 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bb524c-8d03-45c5-ab68-054a9334a881 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:31.207746 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bb524c-8d03-45c5-ab68-054a9334a881 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:31.208027 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bb524c-8d03-45c5-ab68-054a9334a881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:31.212612 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5a55a1-1a0a-495c-be22-27689028464e admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:31.216572 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:31.237198 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 985/1968] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:17:31 2022] PUT /identity/v3/projects/53173c7ff08343adace7682eb5997f94/users/2a5bec8038c64151af84cb1c9d5cfd35/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:31.248425 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5a55a1-1a0a-495c-be22-27689028464e admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:31.252292 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-58bb524c-8d03-45c5-ab68-054a9334a881 None None] MFA Rules not processed for user `8954a22e9f324f52b432801f19215e1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:31.257136 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-58bb524c-8d03-45c5-ab68-054a9334a881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:31.324242 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:31.350048 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 984/1969] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:17:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:31.351214 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5a55a1-1a0a-495c-be22-27689028464e admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:31.351439 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bb524c-8d03-45c5-ab68-054a9334a881 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:31.360602 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d054f864-0ce3-4197-9632-1feb713c2227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:31.362072 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d054f864-0ce3-4197-9632-1feb713c2227 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:31.362072 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d054f864-0ce3-4197-9632-1feb713c2227 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:31.362369 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d054f864-0ce3-4197-9632-1feb713c2227 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:31.362856 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bb524c-8d03-45c5-ab68-054a9334a881 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:31.363354 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d054f864-0ce3-4197-9632-1feb713c2227 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:31.365763 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d054f864-0ce3-4197-9632-1feb713c2227 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:31.368489 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1d2f6c-1851-4775-ae3a-5a0a84b8979f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:31.368877 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d2f6c-1851-4775-ae3a-5a0a84b8979f admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:31.369253 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d2f6c-1851-4775-ae3a-5a0a84b8979f admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:31.369524 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d2f6c-1851-4775-ae3a-5a0a84b8979f admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:31.370836 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1d2f6c-1851-4775-ae3a-5a0a84b8979f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:31.372106 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1d2f6c-1851-4775-ae3a-5a0a84b8979f admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:31.381949 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 986/1970] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:17:31 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:17:31.381949 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e91d3e-a4e8-4cd9-ae70-98d5ec073581 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:31.381949 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e91d3e-a4e8-4cd9-ae70-98d5ec073581 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:31.381949 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e91d3e-a4e8-4cd9-ae70-98d5ec073581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:31.411197 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 985/1971] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:17:31 2022] POST /identity/v3/projects => generated 395 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:17:31.412082 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1d2f6c-1851-4775-ae3a-5a0a84b8979f admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:31.417964 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-f8e91d3e-a4e8-4cd9-ae70-98d5ec073581 None None] MFA Rules not processed for user `2a5bec8038c64151af84cb1c9d5cfd35`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:31.419189 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f8e91d3e-a4e8-4cd9-ae70-98d5ec073581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:31.506843 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1d2f6c-1851-4775-ae3a-5a0a84b8979f admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:31.518387 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 987/1972] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:17:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:31.518387 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e91d3e-a4e8-4cd9-ae70-98d5ec073581 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:31.529622 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631e958a-7001-4d32-8c80-583798e0ec9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:31.530489 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631e958a-7001-4d32-8c80-583798e0ec9f admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:31.530715 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631e958a-7001-4d32-8c80-583798e0ec9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:31.530991 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631e958a-7001-4d32-8c80-583798e0ec9f admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:31.531629 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631e958a-7001-4d32-8c80-583798e0ec9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:31.533661 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631e958a-7001-4d32-8c80-583798e0ec9f admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:31.539055 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e91d3e-a4e8-4cd9-ae70-98d5ec073581 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:31.546415 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1c1445-c4d3-4f5c-b8fc-7154f05eed46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:31.547358 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1c1445-c4d3-4f5c-b8fc-7154f05eed46 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:31.547787 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1c1445-c4d3-4f5c-b8fc-7154f05eed46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:31.555205 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1c1445-c4d3-4f5c-b8fc-7154f05eed46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:31.555205 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1c1445-c4d3-4f5c-b8fc-7154f05eed46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:31.555205 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1c1445-c4d3-4f5c-b8fc-7154f05eed46 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:31.563433 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 988/1973] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:17:31 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:17:31.564040 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1c1445-c4d3-4f5c-b8fc-7154f05eed46 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:31.582840 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1c1445-c4d3-4f5c-b8fc-7154f05eed46 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:31.584969 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 986/1974] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:17:31 2022] POST /identity/v3/projects => generated 375 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:17:31.589801 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631e958a-7001-4d32-8c80-583798e0ec9f admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:31.591693 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-631e958a-7001-4d32-8c80-583798e0ec9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:31.663631 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631e958a-7001-4d32-8c80-583798e0ec9f admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:31.666904 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a033f97-192a-4bf4-a23a-646f2f3d4545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:31.667541 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a033f97-192a-4bf4-a23a-646f2f3d4545 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:31.667818 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a033f97-192a-4bf4-a23a-646f2f3d4545 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:31.668086 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a033f97-192a-4bf4-a23a-646f2f3d4545 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:31.668576 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd6cc4b-771d-4e28-b963-840b043733b7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c163b39e61648c385f065ce8091c3ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:31.668744 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a033f97-192a-4bf4-a23a-646f2f3d4545 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:31.669949 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd6cc4b-771d-4e28-b963-840b043733b7 service glance] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:31.670463 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd6cc4b-771d-4e28-b963-840b043733b7 service glance] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:31.670782 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a033f97-192a-4bf4-a23a-646f2f3d4545 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:31.671302 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd6cc4b-771d-4e28-b963-840b043733b7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:31.693804 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:31.704188 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 989/1975] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:17:31 2022] POST /identity/v3/users => generated 298 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:17:31.705216 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a033f97-192a-4bf4-a23a-646f2f3d4545 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:31.717529 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a033f97-192a-4bf4-a23a-646f2f3d4545 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:31.722376 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f3bd4e-7e60-4e8f-a0e3-c40c756cf9bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:31.722554 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f3bd4e-7e60-4e8f-a0e3-c40c756cf9bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:31.722763 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f3bd4e-7e60-4e8f-a0e3-c40c756cf9bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:31.723091 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f3bd4e-7e60-4e8f-a0e3-c40c756cf9bd admin admin] PATH_INFO: `/v3/projects/78d9ce59960a4fc4be5dcca7a5ae4c22/users/65273d9cc99e4cd491b98bdefd4c66de/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:31.741203 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f3bd4e-7e60-4e8f-a0e3-c40c756cf9bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=78d9ce59960a4fc4be5dcca7a5ae4c22, user_id=65273d9cc99e4cd491b98bdefd4c66de, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:31.743970 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f3bd4e-7e60-4e8f-a0e3-c40c756cf9bd admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:31.750361 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd6cc4b-771d-4e28-b963-840b043733b7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:31.752265 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd6cc4b-771d-4e28-b963-840b043733b7 service glance] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:31.761495 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 990/1976] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:17:31 2022] PUT /identity/v3/projects/78d9ce59960a4fc4be5dcca7a5ae4c22/users/65273d9cc99e4cd491b98bdefd4c66de/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:31.762825 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f3bd4e-7e60-4e8f-a0e3-c40c756cf9bd admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:31.776673 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 987/1977] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:17:31 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:17:31.778021 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd6cc4b-771d-4e28-b963-840b043733b7 service glance] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:31.788704 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:31.799456 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd6cc4b-771d-4e28-b963-840b043733b7 service glance] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:31.922236 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb57e35-983c-4029-bab9-44827aa594c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:31.922886 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb57e35-983c-4029-bab9-44827aa594c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:31.923167 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb57e35-983c-4029-bab9-44827aa594c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:31.923436 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb57e35-983c-4029-bab9-44827aa594c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:31.927772 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb57e35-983c-4029-bab9-44827aa594c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:31.930231 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb57e35-983c-4029-bab9-44827aa594c6 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:31.937810 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f3bd4e-7e60-4e8f-a0e3-c40c756cf9bd admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:31.944554 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda8f86a-d0c9-4fc6-a0eb-765def70385a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:31.945536 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda8f86a-d0c9-4fc6-a0eb-765def70385a admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:31.945536 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda8f86a-d0c9-4fc6-a0eb-765def70385a admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:31.945831 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda8f86a-d0c9-4fc6-a0eb-765def70385a admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:31.946420 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda8f86a-d0c9-4fc6-a0eb-765def70385a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:31.948869 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda8f86a-d0c9-4fc6-a0eb-765def70385a admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:31.970481 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 988/1978] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:17:31 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:17:31.975952 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8cc51d-3937-45e2-8ce3-e3b54c25d664 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:31.976151 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8cc51d-3937-45e2-8ce3-e3b54c25d664 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:31.976377 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8cc51d-3937-45e2-8ce3-e3b54c25d664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:31.982930 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 991/1979] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:17:31 2022] POST /identity/v3/users => generated 294 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:17:31.984304 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda8f86a-d0c9-4fc6-a0eb-765def70385a admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:31.997452 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda8f86a-d0c9-4fc6-a0eb-765def70385a admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:32.007962 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204c0af1-f912-4e90-a933-2beba894da97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:32.009279 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204c0af1-f912-4e90-a933-2beba894da97 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:32.009939 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204c0af1-f912-4e90-a933-2beba894da97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:32.010550 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204c0af1-f912-4e90-a933-2beba894da97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:32.013079 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204c0af1-f912-4e90-a933-2beba894da97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:32.016371 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204c0af1-f912-4e90-a933-2beba894da97 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:32.033707 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 992/1980] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:17:31 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:17:32.033707 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204c0af1-f912-4e90-a933-2beba894da97 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:32.057508 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204c0af1-f912-4e90-a933-2beba894da97 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:32.071984 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759941e3-ce5d-4af8-b88b-7c1ff81dc9cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:32.074234 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759941e3-ce5d-4af8-b88b-7c1ff81dc9cb admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:32.075090 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759941e3-ce5d-4af8-b88b-7c1ff81dc9cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:32.076215 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759941e3-ce5d-4af8-b88b-7c1ff81dc9cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:32.078104 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759941e3-ce5d-4af8-b88b-7c1ff81dc9cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:32.078476 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-2f8cc51d-3937-45e2-8ce3-e3b54c25d664 None None] MFA Rules not processed for user `65273d9cc99e4cd491b98bdefd4c66de`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:32.079719 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-2f8cc51d-3937-45e2-8ce3-e3b54c25d664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:32.082472 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759941e3-ce5d-4af8-b88b-7c1ff81dc9cb admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:32.121246 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 993/1981] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:17:32 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:17:32.121246 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759941e3-ce5d-4af8-b88b-7c1ff81dc9cb admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:32.168090 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759941e3-ce5d-4af8-b88b-7c1ff81dc9cb admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:32.180708 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aaf101-18fb-4ef6-be1f-312a8c1e15b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:32.181888 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aaf101-18fb-4ef6-be1f-312a8c1e15b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:32.184006 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aaf101-18fb-4ef6-be1f-312a8c1e15b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:32.184006 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aaf101-18fb-4ef6-be1f-312a8c1e15b8 admin admin] PATH_INFO: `/v3/projects/876a7d0ff0ec420d98d2f5711cbc4548/users/4171df63bd70454fa4b9abc736d7fc3e/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:32.228348 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aaf101-18fb-4ef6-be1f-312a8c1e15b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=876a7d0ff0ec420d98d2f5711cbc4548, user_id=4171df63bd70454fa4b9abc736d7fc3e, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:32.233830 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aaf101-18fb-4ef6-be1f-312a8c1e15b8 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:32.262145 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 994/1982] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:17:32 2022] PUT /identity/v3/projects/876a7d0ff0ec420d98d2f5711cbc4548/users/4171df63bd70454fa4b9abc736d7fc3e/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:32.263631 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aaf101-18fb-4ef6-be1f-312a8c1e15b8 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:32.280996 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 989/1983] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:17:31 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:32.280996 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c0e7d0-9fdc-4940-b705-8b72fbbdde51 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:32.280996 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c0e7d0-9fdc-4940-b705-8b72fbbdde51 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:32.280996 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c0e7d0-9fdc-4940-b705-8b72fbbdde51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:32.318138 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-60c0e7d0-9fdc-4940-b705-8b72fbbdde51 None None] MFA Rules not processed for user `4171df63bd70454fa4b9abc736d7fc3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:32.319725 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-60c0e7d0-9fdc-4940-b705-8b72fbbdde51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:32.414138 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aaf101-18fb-4ef6-be1f-312a8c1e15b8 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:32.425492 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2409bc-818f-498b-a6f3-dd0c2bc974a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:32.426672 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2409bc-818f-498b-a6f3-dd0c2bc974a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:32.427104 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2409bc-818f-498b-a6f3-dd0c2bc974a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:32.427517 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2409bc-818f-498b-a6f3-dd0c2bc974a3 admin admin] PATH_INFO: `/v3/projects/646c9afc62954c9bad1b81c6920a7663/users/d77803c6aa7b4a1a8ece9714d8d5e0b4/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:32.442384 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 990/1984] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:17:32 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:32.442384 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c0e7d0-9fdc-4940-b705-8b72fbbdde51 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:32.458055 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2409bc-818f-498b-a6f3-dd0c2bc974a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=646c9afc62954c9bad1b81c6920a7663, user_id=d77803c6aa7b4a1a8ece9714d8d5e0b4, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:32.461082 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c0e7d0-9fdc-4940-b705-8b72fbbdde51 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:32.462314 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2409bc-818f-498b-a6f3-dd0c2bc974a3 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:32.467992 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e597d46-573d-44d7-bded-cb8dce668192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:32.468807 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e597d46-573d-44d7-bded-cb8dce668192 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:32.469231 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e597d46-573d-44d7-bded-cb8dce668192 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:32.469596 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e597d46-573d-44d7-bded-cb8dce668192 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:32.470541 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e597d46-573d-44d7-bded-cb8dce668192 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:32.473880 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e597d46-573d-44d7-bded-cb8dce668192 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:32.482341 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 995/1985] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:17:32 2022] PUT /identity/v3/projects/646c9afc62954c9bad1b81c6920a7663/users/d77803c6aa7b4a1a8ece9714d8d5e0b4/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:32.485682 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2409bc-818f-498b-a6f3-dd0c2bc974a3 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:32.487723 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-3f2409bc-818f-498b-a6f3-dd0c2bc974a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:32.493364 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:32.496859 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:32.519008 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 991/1986] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:17:32 2022] POST /identity/v3/projects => generated 401 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:17:32.520888 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e597d46-573d-44d7-bded-cb8dce668192 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:32.615630 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2409bc-818f-498b-a6f3-dd0c2bc974a3 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:32.666832 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e597d46-573d-44d7-bded-cb8dce668192 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:32.675063 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c065e753-a802-4515-b536-b2074e2a5b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:32.675527 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c065e753-a802-4515-b536-b2074e2a5b0d admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:32.676050 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c065e753-a802-4515-b536-b2074e2a5b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:32.676499 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c065e753-a802-4515-b536-b2074e2a5b0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:32.677667 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c065e753-a802-4515-b536-b2074e2a5b0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:32.680600 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c065e753-a802-4515-b536-b2074e2a5b0d admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:32.702226 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 992/1987] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:17:32 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:17:32.703798 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c065e753-a802-4515-b536-b2074e2a5b0d admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:32.721825 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c065e753-a802-4515-b536-b2074e2a5b0d admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:32.722983 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a73ae17-88a5-4e9d-8259-9916dbdbb0ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:32.723693 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a73ae17-88a5-4e9d-8259-9916dbdbb0ea service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:32.724075 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a73ae17-88a5-4e9d-8259-9916dbdbb0ea service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:32.724528 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a73ae17-88a5-4e9d-8259-9916dbdbb0ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:32.731192 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd6c387-bbad-4c1d-a175-db1d6269ec04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:32.732572 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd6c387-bbad-4c1d-a175-db1d6269ec04 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:32.733124 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd6c387-bbad-4c1d-a175-db1d6269ec04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:32.733825 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd6c387-bbad-4c1d-a175-db1d6269ec04 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:32.734950 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd6c387-bbad-4c1d-a175-db1d6269ec04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:32.738855 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd6c387-bbad-4c1d-a175-db1d6269ec04 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:32.772192 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 993/1988] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:17:32 2022] POST /identity/v3/users => generated 307 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:17:32.773737 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd6c387-bbad-4c1d-a175-db1d6269ec04 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:32.797604 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd6c387-bbad-4c1d-a175-db1d6269ec04 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:32.808171 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1c4dcb-8079-4787-8945-139f4c76b0a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:32.808746 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1c4dcb-8079-4787-8945-139f4c76b0a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:32.809050 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1c4dcb-8079-4787-8945-139f4c76b0a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:32.809325 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1c4dcb-8079-4787-8945-139f4c76b0a0 admin admin] PATH_INFO: `/v3/projects/646c9afc62954c9bad1b81c6920a7663/users/d77803c6aa7b4a1a8ece9714d8d5e0b4/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:32.811341 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1c4dcb-8079-4787-8945-139f4c76b0a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=646c9afc62954c9bad1b81c6920a7663, user_id=d77803c6aa7b4a1a8ece9714d8d5e0b4, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:32.814912 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1c4dcb-8079-4787-8945-139f4c76b0a0 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:32.826343 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a73ae17-88a5-4e9d-8259-9916dbdbb0ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:32.827644 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 994/1989] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:17:32 2022] PUT /identity/v3/projects/646c9afc62954c9bad1b81c6920a7663/users/d77803c6aa7b4a1a8ece9714d8d5e0b4/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:32.828498 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1c4dcb-8079-4787-8945-139f4c76b0a0 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:32.828605 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a73ae17-88a5-4e9d-8259-9916dbdbb0ea service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:32.847581 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 996/1990] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:17:32 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:17:32.851045 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39099e88-6316-4a5c-977c-83ac2f43ef61 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:32.851751 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39099e88-6316-4a5c-977c-83ac2f43ef61 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:32.852315 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39099e88-6316-4a5c-977c-83ac2f43ef61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:32.897038 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-39099e88-6316-4a5c-977c-83ac2f43ef61 None None] MFA Rules not processed for user `d77803c6aa7b4a1a8ece9714d8d5e0b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:32.898488 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-39099e88-6316-4a5c-977c-83ac2f43ef61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:32.985385 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1c4dcb-8079-4787-8945-139f4c76b0a0 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:32.989427 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:33.000515 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 997/1991] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:17:32 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:33.002999 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bd1536-5dab-449f-9fc5-9ba41618971e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:33.003766 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bd1536-5dab-449f-9fc5-9ba41618971e admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:33.004222 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bd1536-5dab-449f-9fc5-9ba41618971e admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:33.004811 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bd1536-5dab-449f-9fc5-9ba41618971e admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:33.005829 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bd1536-5dab-449f-9fc5-9ba41618971e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:33.007917 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bd1536-5dab-449f-9fc5-9ba41618971e admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:33.027992 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39099e88-6316-4a5c-977c-83ac2f43ef61 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:33.046136 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 995/1992] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:17:32 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:17:33.050850 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bd1536-5dab-449f-9fc5-9ba41618971e admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:33.084265 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bd1536-5dab-449f-9fc5-9ba41618971e admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:33.095444 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c913157-ead2-423e-96af-4b8d58484d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:33.096199 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c913157-ead2-423e-96af-4b8d58484d00 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:33.096587 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c913157-ead2-423e-96af-4b8d58484d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:33.096949 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c913157-ead2-423e-96af-4b8d58484d00 admin admin] PATH_INFO: `/v3/projects/c33d7811851f4218b8fb7d4cfa0c3b96/users/b873f1e4cacf4820840a0a52766c666e/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:33.125173 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c913157-ead2-423e-96af-4b8d58484d00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c33d7811851f4218b8fb7d4cfa0c3b96, user_id=b873f1e4cacf4820840a0a52766c666e, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:33.129055 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c913157-ead2-423e-96af-4b8d58484d00 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:33.154711 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 996/1993] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:17:33 2022] PUT /identity/v3/projects/c33d7811851f4218b8fb7d4cfa0c3b96/users/b873f1e4cacf4820840a0a52766c666e/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:33.160654 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c913157-ead2-423e-96af-4b8d58484d00 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:33.181633 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39099e88-6316-4a5c-977c-83ac2f43ef61 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:33.261271 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:33.325709 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4d01af-e733-4331-ae48-9d4664eefe46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:33.329440 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4d01af-e733-4331-ae48-9d4664eefe46 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:33.329842 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4d01af-e733-4331-ae48-9d4664eefe46 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:33.331219 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4d01af-e733-4331-ae48-9d4664eefe46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:33.336384 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-3e4d01af-e733-4331-ae48-9d4664eefe46 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:33.356872 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c913157-ead2-423e-96af-4b8d58484d00 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:33.374432 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0d726d-c5e7-4977-b893-4992c3bcd29d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:33.375806 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d726d-c5e7-4977-b893-4992c3bcd29d admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:33.376459 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d726d-c5e7-4977-b893-4992c3bcd29d admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:33.377187 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d726d-c5e7-4977-b893-4992c3bcd29d admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:33.378650 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0d726d-c5e7-4977-b893-4992c3bcd29d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:33.381877 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0d726d-c5e7-4977-b893-4992c3bcd29d admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:33.428223 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 997/1994] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:17:33 2022] GET /identity/v3/roles => generated 1420 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:17:33.435313 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0d726d-c5e7-4977-b893-4992c3bcd29d admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:33.475643 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0d726d-c5e7-4977-b893-4992c3bcd29d admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:33.492299 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae385ff5-601d-45ad-9b25-f0879ee834a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:33.493482 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae385ff5-601d-45ad-9b25-f0879ee834a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:33.494679 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae385ff5-601d-45ad-9b25-f0879ee834a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:33.496048 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae385ff5-601d-45ad-9b25-f0879ee834a8 admin admin] PATH_INFO: `/v3/projects/c33d7811851f4218b8fb7d4cfa0c3b96/users/b873f1e4cacf4820840a0a52766c666e/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:33.499885 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae385ff5-601d-45ad-9b25-f0879ee834a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c33d7811851f4218b8fb7d4cfa0c3b96, user_id=b873f1e4cacf4820840a0a52766c666e, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:33.506365 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae385ff5-601d-45ad-9b25-f0879ee834a8 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:33.517509 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4d01af-e733-4331-ae48-9d4664eefe46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:33.521640 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4d01af-e733-4331-ae48-9d4664eefe46 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:33.533881 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 998/1995] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:17:33 2022] PUT /identity/v3/projects/c33d7811851f4218b8fb7d4cfa0c3b96/users/b873f1e4cacf4820840a0a52766c666e/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:33.552303 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a53e41a-9e7c-4891-a02a-25432e9bd362 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:33.553414 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a53e41a-9e7c-4891-a02a-25432e9bd362 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:33.553867 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a53e41a-9e7c-4891-a02a-25432e9bd362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:33.569048 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 998/1996] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:17:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:17:33.661835 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-1a53e41a-9e7c-4891-a02a-25432e9bd362 None None] MFA Rules not processed for user `b873f1e4cacf4820840a0a52766c666e`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:33.663138 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1a53e41a-9e7c-4891-a02a-25432e9bd362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:33.886388 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 999/1997] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:17:33 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:33.975655 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4d01af-e733-4331-ae48-9d4664eefe46 service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:33.977353 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-3e4d01af-e733-4331-ae48-9d4664eefe46 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:34.007665 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4d01af-e733-4331-ae48-9d4664eefe46 service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:34.092988 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3160c3ae-ce1c-4bd5-af42-c5a8ad8b928c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:34.093995 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3160c3ae-ce1c-4bd5-af42-c5a8ad8b928c service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:34.094367 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3160c3ae-ce1c-4bd5-af42-c5a8ad8b928c service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:34.094762 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3160c3ae-ce1c-4bd5-af42-c5a8ad8b928c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:34.106818 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3160c3ae-ce1c-4bd5-af42-c5a8ad8b928c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:34.108534 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3160c3ae-ce1c-4bd5-af42-c5a8ad8b928c service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:34.121252 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 999/1998] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:17:33 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:17:34.236428 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a53e41a-9e7c-4891-a02a-25432e9bd362 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:34.284453 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a53e41a-9e7c-4891-a02a-25432e9bd362 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:34.289474 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ee7a44-bf7f-4477-b7d6-7efc50343eeb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:34.290271 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ee7a44-bf7f-4477-b7d6-7efc50343eeb service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:34.290848 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ee7a44-bf7f-4477-b7d6-7efc50343eeb service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:34.291315 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ee7a44-bf7f-4477-b7d6-7efc50343eeb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:34.362530 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ee7a44-bf7f-4477-b7d6-7efc50343eeb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:34.364028 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ee7a44-bf7f-4477-b7d6-7efc50343eeb service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:34.374105 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1000/1999] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:17:34 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:17:36.580742 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3160c3ae-ce1c-4bd5-af42-c5a8ad8b928c service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:36.589787 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3160c3ae-ce1c-4bd5-af42-c5a8ad8b928c service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:36.594101 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0f4665-842f-4884-a578-84df7eb6455a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:36.594799 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0f4665-842f-4884-a578-84df7eb6455a service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:36.595143 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0f4665-842f-4884-a578-84df7eb6455a service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:36.595865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0f4665-842f-4884-a578-84df7eb6455a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:36.605712 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:36.660983 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0f4665-842f-4884-a578-84df7eb6455a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:36.662413 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0f4665-842f-4884-a578-84df7eb6455a service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:36.671864 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1000/2000] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:17:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:17:38.730141 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ee7a44-bf7f-4477-b7d6-7efc50343eeb service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:38.745499 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ee7a44-bf7f-4477-b7d6-7efc50343eeb service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:38.745499 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bf031a-2894-4d41-8ffd-6dc3faae1917 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:38.745499 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bf031a-2894-4d41-8ffd-6dc3faae1917 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:38.745499 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bf031a-2894-4d41-8ffd-6dc3faae1917 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:38.745994 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bf031a-2894-4d41-8ffd-6dc3faae1917 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:38.790250 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:38.815944 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bf031a-2894-4d41-8ffd-6dc3faae1917 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:38.817493 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bf031a-2894-4d41-8ffd-6dc3faae1917 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:38.827350 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1001/2001] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:17:38 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:17:42.978777 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0f4665-842f-4884-a578-84df7eb6455a service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:43.050081 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0f4665-842f-4884-a578-84df7eb6455a service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:43.054835 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5499767d-7fa0-4902-a036-f4cea5fb0857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:43.055481 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5499767d-7fa0-4902-a036-f4cea5fb0857 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:43.055757 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5499767d-7fa0-4902-a036-f4cea5fb0857 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:43.056095 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5499767d-7fa0-4902-a036-f4cea5fb0857 admin admin] PATH_INFO: `/v3/users/65273d9cc99e4cd491b98bdefd4c66de` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:43.057217 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5499767d-7fa0-4902-a036-f4cea5fb0857 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65273d9cc99e4cd491b98bdefd4c66de)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:43.059251 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5499767d-7fa0-4902-a036-f4cea5fb0857 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:43.120338 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-5499767d-7fa0-4902-a036-f4cea5fb0857 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65273d9cc99e4cd491b98bdefd4c66de'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:43.121081 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-5499767d-7fa0-4902-a036-f4cea5fb0857 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65273d9cc99e4cd491b98bdefd4c66de'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:43.130766 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-5499767d-7fa0-4902-a036-f4cea5fb0857 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65273d9cc99e4cd491b98bdefd4c66de'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:43.147564 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-5499767d-7fa0-4902-a036-f4cea5fb0857 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65273d9cc99e4cd491b98bdefd4c66de'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:43.160156 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-5499767d-7fa0-4902-a036-f4cea5fb0857 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65273d9cc99e4cd491b98bdefd4c66de'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:43.167212 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1001/2002] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:17:42 2022] DELETE /identity/v3/users/65273d9cc99e4cd491b98bdefd4c66de => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:43.171966 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebc07b4-14d5-4303-95c8-e1e48a9ba517 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:43.172659 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebc07b4-14d5-4303-95c8-e1e48a9ba517 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:43.172659 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebc07b4-14d5-4303-95c8-e1e48a9ba517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:43.201801 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-cebc07b4-14d5-4303-95c8-e1e48a9ba517 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:43.203100 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-cebc07b4-14d5-4303-95c8-e1e48a9ba517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:43.277078 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1002/2003] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:17:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:43.285137 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5499767d-7fa0-4902-a036-f4cea5fb0857 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:43.286720 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5499767d-7fa0-4902-a036-f4cea5fb0857 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:43.320570 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:43.351018 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5499767d-7fa0-4902-a036-f4cea5fb0857 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:43.355258 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5bb44f-fefa-41f9-b635-ad79deac56b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:43.355880 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5bb44f-fefa-41f9-b635-ad79deac56b5 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:43.356155 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5bb44f-fefa-41f9-b635-ad79deac56b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:43.356425 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5bb44f-fefa-41f9-b635-ad79deac56b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:43.366430 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5bb44f-fefa-41f9-b635-ad79deac56b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:43.367875 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5bb44f-fefa-41f9-b635-ad79deac56b5 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:43.379033 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1002/2004] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:17:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:17:43.772589 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebc07b4-14d5-4303-95c8-e1e48a9ba517 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:43.774220 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-cebc07b4-14d5-4303-95c8-e1e48a9ba517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:43.789063 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebc07b4-14d5-4303-95c8-e1e48a9ba517 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:43.797126 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2dd45c-c4b8-44e4-8ac6-b744dcb6f804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:43.800550 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2dd45c-c4b8-44e4-8ac6-b744dcb6f804 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:43.801172 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2dd45c-c4b8-44e4-8ac6-b744dcb6f804 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:43.801732 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2dd45c-c4b8-44e4-8ac6-b744dcb6f804 admin admin] PATH_INFO: `/v3/projects/78d9ce59960a4fc4be5dcca7a5ae4c22` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:43.804017 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2dd45c-c4b8-44e4-8ac6-b744dcb6f804 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78d9ce59960a4fc4be5dcca7a5ae4c22)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:43.806341 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2dd45c-c4b8-44e4-8ac6-b744dcb6f804 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:43.875146 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ca2dd45c-c4b8-44e4-8ac6-b744dcb6f804 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78d9ce59960a4fc4be5dcca7a5ae4c22', 'request_id': 'req-ca2dd45c-c4b8-44e4-8ac6-b744dcb6f804'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:43.882001 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ca2dd45c-c4b8-44e4-8ac6-b744dcb6f804 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78d9ce59960a4fc4be5dcca7a5ae4c22', 'request_id': 'req-ca2dd45c-c4b8-44e4-8ac6-b744dcb6f804'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:43.891751 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ca2dd45c-c4b8-44e4-8ac6-b744dcb6f804 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78d9ce59960a4fc4be5dcca7a5ae4c22', 'request_id': 'req-ca2dd45c-c4b8-44e4-8ac6-b744dcb6f804'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:43.906422 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ca2dd45c-c4b8-44e4-8ac6-b744dcb6f804 admin admin] The token cache is being invalidate because project 78d9ce59960a4fc4be5dcca7a5ae4c22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:17:43.906766 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ca2dd45c-c4b8-44e4-8ac6-b744dcb6f804 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:43.907560 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ca2dd45c-c4b8-44e4-8ac6-b744dcb6f804 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:43.909267 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1003/2005] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:17:43 2022] DELETE /identity/v3/projects/78d9ce59960a4fc4be5dcca7a5ae4c22 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:43.913787 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5bb44f-fefa-41f9-b635-ad79deac56b5 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:43.924575 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5bb44f-fefa-41f9-b635-ad79deac56b5 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:43.994271 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5825f0-000a-40c4-82a6-5b3aab691328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:43.995448 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5825f0-000a-40c4-82a6-5b3aab691328 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:43.996008 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5825f0-000a-40c4-82a6-5b3aab691328 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:43.996521 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5825f0-000a-40c4-82a6-5b3aab691328 admin admin] PATH_INFO: `/v3/users/b873f1e4cacf4820840a0a52766c666e` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:43.998465 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5825f0-000a-40c4-82a6-5b3aab691328 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b873f1e4cacf4820840a0a52766c666e)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:44.005096 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5825f0-000a-40c4-82a6-5b3aab691328 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:44.068977 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-ba5825f0-000a-40c4-82a6-5b3aab691328 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b873f1e4cacf4820840a0a52766c666e'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:44.069723 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-ba5825f0-000a-40c4-82a6-5b3aab691328 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b873f1e4cacf4820840a0a52766c666e'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:44.077046 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-ba5825f0-000a-40c4-82a6-5b3aab691328 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b873f1e4cacf4820840a0a52766c666e'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:44.090199 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-ba5825f0-000a-40c4-82a6-5b3aab691328 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b873f1e4cacf4820840a0a52766c666e'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:44.098807 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-ba5825f0-000a-40c4-82a6-5b3aab691328 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b873f1e4cacf4820840a0a52766c666e'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:44.105296 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1003/2006] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:17:43 2022] DELETE /identity/v3/users/b873f1e4cacf4820840a0a52766c666e => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:17:44.441970 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2dd45c-c4b8-44e4-8ac6-b744dcb6f804 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:44.444859 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ca2dd45c-c4b8-44e4-8ac6-b744dcb6f804 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:44.468216 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2dd45c-c4b8-44e4-8ac6-b744dcb6f804 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:44.554493 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:44.562341 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e751113b-694f-4195-871e-ed7dae40e444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:44.563332 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e751113b-694f-4195-871e-ed7dae40e444 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:44.563675 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e751113b-694f-4195-871e-ed7dae40e444 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:44.564008 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e751113b-694f-4195-871e-ed7dae40e444 admin admin] PATH_INFO: `/v3/projects/c33d7811851f4218b8fb7d4cfa0c3b96` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:44.566091 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e751113b-694f-4195-871e-ed7dae40e444 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c33d7811851f4218b8fb7d4cfa0c3b96)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:44.568688 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e751113b-694f-4195-871e-ed7dae40e444 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:44.685304 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e751113b-694f-4195-871e-ed7dae40e444 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c33d7811851f4218b8fb7d4cfa0c3b96', 'request_id': 'req-e751113b-694f-4195-871e-ed7dae40e444'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:44.692949 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e751113b-694f-4195-871e-ed7dae40e444 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c33d7811851f4218b8fb7d4cfa0c3b96', 'request_id': 'req-e751113b-694f-4195-871e-ed7dae40e444'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:44.714938 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e751113b-694f-4195-871e-ed7dae40e444 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c33d7811851f4218b8fb7d4cfa0c3b96', 'request_id': 'req-e751113b-694f-4195-871e-ed7dae40e444'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:44.738584 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e751113b-694f-4195-871e-ed7dae40e444 admin admin] The token cache is being invalidate because project c33d7811851f4218b8fb7d4cfa0c3b96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:17:44.738848 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e751113b-694f-4195-871e-ed7dae40e444 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:44.739682 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e751113b-694f-4195-871e-ed7dae40e444 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:44.741959 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1004/2007] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:17:44 2022] DELETE /identity/v3/projects/c33d7811851f4218b8fb7d4cfa0c3b96 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:44.747669 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26fb3ce-01ec-4135-ba52-de4b2ea00422 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:44.748126 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26fb3ce-01ec-4135-ba52-de4b2ea00422 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:44.748545 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26fb3ce-01ec-4135-ba52-de4b2ea00422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:44.808133 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-b26fb3ce-01ec-4135-ba52-de4b2ea00422 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:44.809762 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b26fb3ce-01ec-4135-ba52-de4b2ea00422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:44.871538 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:44.904801 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1004/2008] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:17:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:44.913384 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6624d71-4cee-4d3d-bfc8-407742f9b1d0 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:44.913819 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6624d71-4cee-4d3d-bfc8-407742f9b1d0 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:44.914248 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6624d71-4cee-4d3d-bfc8-407742f9b1d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:44.920292 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26fb3ce-01ec-4135-ba52-de4b2ea00422 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:44.927728 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b26fb3ce-01ec-4135-ba52-de4b2ea00422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:44.946611 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26fb3ce-01ec-4135-ba52-de4b2ea00422 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:44.952190 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4596d39e-015f-457f-9475-9d93f7b0e60b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:44.952886 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4596d39e-015f-457f-9475-9d93f7b0e60b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:44.953161 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4596d39e-015f-457f-9475-9d93f7b0e60b admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:44.953437 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4596d39e-015f-457f-9475-9d93f7b0e60b admin admin] PATH_INFO: `/v3/users/4171df63bd70454fa4b9abc736d7fc3e` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:44.954616 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4596d39e-015f-457f-9475-9d93f7b0e60b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4171df63bd70454fa4b9abc736d7fc3e)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:44.956582 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4596d39e-015f-457f-9475-9d93f7b0e60b admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:44.962392 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-b6624d71-4cee-4d3d-bfc8-407742f9b1d0 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:44.964522 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b6624d71-4cee-4d3d-bfc8-407742f9b1d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:44.982818 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1005/2009] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:17:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:44.987793 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6624d71-4cee-4d3d-bfc8-407742f9b1d0 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:45.004159 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6624d71-4cee-4d3d-bfc8-407742f9b1d0 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:45.013389 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2914c6d5-dd68-4cf5-a247-0ff9f88897d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:45.014517 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2914c6d5-dd68-4cf5-a247-0ff9f88897d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:45.015127 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2914c6d5-dd68-4cf5-a247-0ff9f88897d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:45.015593 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2914c6d5-dd68-4cf5-a247-0ff9f88897d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:45.016334 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2914c6d5-dd68-4cf5-a247-0ff9f88897d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:17:45.016637 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2914c6d5-dd68-4cf5-a247-0ff9f88897d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:45.018299 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2914c6d5-dd68-4cf5-a247-0ff9f88897d7 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:45.035437 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1006/2010] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 6 16:17:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:17:45.035677 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-4596d39e-015f-457f-9475-9d93f7b0e60b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4171df63bd70454fa4b9abc736d7fc3e'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:45.035677 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-4596d39e-015f-457f-9475-9d93f7b0e60b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4171df63bd70454fa4b9abc736d7fc3e'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:45.036634 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2914c6d5-dd68-4cf5-a247-0ff9f88897d7 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:45.049437 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2914c6d5-dd68-4cf5-a247-0ff9f88897d7 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:45.055228 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9762f4b7-b884-411d-9c62-5d5634275432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:45.056141 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9762f4b7-b884-411d-9c62-5d5634275432 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:45.056604 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9762f4b7-b884-411d-9c62-5d5634275432 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:45.057236 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9762f4b7-b884-411d-9c62-5d5634275432 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:45.058496 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9762f4b7-b884-411d-9c62-5d5634275432 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:45.060597 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-4596d39e-015f-457f-9475-9d93f7b0e60b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4171df63bd70454fa4b9abc736d7fc3e'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:45.061096 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9762f4b7-b884-411d-9c62-5d5634275432 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:45.080910 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-4596d39e-015f-457f-9475-9d93f7b0e60b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4171df63bd70454fa4b9abc736d7fc3e'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:45.100064 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-4596d39e-015f-457f-9475-9d93f7b0e60b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4171df63bd70454fa4b9abc736d7fc3e'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:45.100881 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1007/2011] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:17:45 2022] POST /identity/v3/projects => generated 383 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:17:45.105733 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9762f4b7-b884-411d-9c62-5d5634275432 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:45.108435 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-9762f4b7-b884-411d-9c62-5d5634275432 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:45.121763 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1005/2012] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:17:44 2022] DELETE /identity/v3/users/4171df63bd70454fa4b9abc736d7fc3e => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:45.123205 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f812d7-ffa8-4cb6-a697-7c4cdff60585 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:45.123495 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f812d7-ffa8-4cb6-a697-7c4cdff60585 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:45.123785 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f812d7-ffa8-4cb6-a697-7c4cdff60585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:45.197885 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-10f812d7-ffa8-4cb6-a697-7c4cdff60585 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:45.199359 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-10f812d7-ffa8-4cb6-a697-7c4cdff60585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:45.235107 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9762f4b7-b884-411d-9c62-5d5634275432 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:45.299016 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:45.307131 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1006/2013] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:17:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:45.311831 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932619b6-44d2-4f1d-a70c-641b0fa92154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:45.319737 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932619b6-44d2-4f1d-a70c-641b0fa92154 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:45.319943 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f812d7-ffa8-4cb6-a697-7c4cdff60585 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:45.321467 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932619b6-44d2-4f1d-a70c-641b0fa92154 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:45.321832 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932619b6-44d2-4f1d-a70c-641b0fa92154 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:45.322343 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-10f812d7-ffa8-4cb6-a697-7c4cdff60585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:45.322610 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932619b6-44d2-4f1d-a70c-641b0fa92154 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:45.325116 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932619b6-44d2-4f1d-a70c-641b0fa92154 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:45.355370 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1008/2014] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:17:45 2022] POST /identity/v3/users => generated 292 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:17:45.356597 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932619b6-44d2-4f1d-a70c-641b0fa92154 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:45.374629 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932619b6-44d2-4f1d-a70c-641b0fa92154 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:45.381055 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b451f9-e00e-4d40-9369-196ce4341913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:45.381686 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b451f9-e00e-4d40-9369-196ce4341913 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:45.382026 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b451f9-e00e-4d40-9369-196ce4341913 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:45.382301 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b451f9-e00e-4d40-9369-196ce4341913 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:45.383567 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b451f9-e00e-4d40-9369-196ce4341913 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:45.385317 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b451f9-e00e-4d40-9369-196ce4341913 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:45.397162 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f812d7-ffa8-4cb6-a697-7c4cdff60585 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:45.398907 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1009/2015] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:17:45 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:17:45.402129 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3baf3f5-9631-4d89-b0c2-f25932e90117 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:45.402773 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3baf3f5-9631-4d89-b0c2-f25932e90117 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:45.403059 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3baf3f5-9631-4d89-b0c2-f25932e90117 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:45.403280 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b451f9-e00e-4d40-9369-196ce4341913 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:45.403404 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3baf3f5-9631-4d89-b0c2-f25932e90117 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:45.413835 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3baf3f5-9631-4d89-b0c2-f25932e90117 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:45.414128 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b451f9-e00e-4d40-9369-196ce4341913 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:45.416069 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3baf3f5-9631-4d89-b0c2-f25932e90117 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:45.419839 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2220ea71-83fe-4ab2-9567-c4668cf03309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:45.420552 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2220ea71-83fe-4ab2-9567-c4668cf03309 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:45.420876 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2220ea71-83fe-4ab2-9567-c4668cf03309 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:45.421226 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2220ea71-83fe-4ab2-9567-c4668cf03309 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:45.422086 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2220ea71-83fe-4ab2-9567-c4668cf03309 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:45.423921 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2220ea71-83fe-4ab2-9567-c4668cf03309 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:45.428589 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1007/2016] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:17:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:17:45.437603 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1010/2017] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:17:45 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:17:45.443834 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3baf3f5-9631-4d89-b0c2-f25932e90117 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:45.459978 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3baf3f5-9631-4d89-b0c2-f25932e90117 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:45.464844 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5a14e5-f0f8-4078-b46b-a4f71db18e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:45.465516 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a14e5-f0f8-4078-b46b-a4f71db18e6e admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:45.465818 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a14e5-f0f8-4078-b46b-a4f71db18e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:45.466087 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a14e5-f0f8-4078-b46b-a4f71db18e6e admin admin] PATH_INFO: `/v3/projects/f6e89f64f5404bda96cf64d754cbcbb2/users/d3baa1fea74144b5b170613eedfe44ae/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:45.469443 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:45.486589 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5a14e5-f0f8-4078-b46b-a4f71db18e6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6e89f64f5404bda96cf64d754cbcbb2, user_id=d3baa1fea74144b5b170613eedfe44ae, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:45.492091 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5a14e5-f0f8-4078-b46b-a4f71db18e6e admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:45.507812 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1008/2018] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:17:45 2022] PUT /identity/v3/projects/f6e89f64f5404bda96cf64d754cbcbb2/users/d3baa1fea74144b5b170613eedfe44ae/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:45.518873 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b679c9-ea82-4c31-81e8-11e4ff1979fc None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:45.519553 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b679c9-ea82-4c31-81e8-11e4ff1979fc None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:45.519553 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b679c9-ea82-4c31-81e8-11e4ff1979fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:45.550700 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-09b679c9-ea82-4c31-81e8-11e4ff1979fc None None] MFA Rules not processed for user `d3baa1fea74144b5b170613eedfe44ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:45.551960 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-09b679c9-ea82-4c31-81e8-11e4ff1979fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:45.635343 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1011/2019] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:17:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:45.639864 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5a14e5-f0f8-4078-b46b-a4f71db18e6e admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:45.708079 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5a14e5-f0f8-4078-b46b-a4f71db18e6e admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:45.713007 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6298bd55-35da-4c79-ba5f-f6dc47cf416f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:45.713494 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6298bd55-35da-4c79-ba5f-f6dc47cf416f admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:45.713935 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6298bd55-35da-4c79-ba5f-f6dc47cf416f admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:45.714212 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6298bd55-35da-4c79-ba5f-f6dc47cf416f admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:45.714869 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6298bd55-35da-4c79-ba5f-f6dc47cf416f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:45.716837 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6298bd55-35da-4c79-ba5f-f6dc47cf416f admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:45.751162 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1009/2020] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:17:45 2022] POST /identity/v3/projects => generated 383 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:17:45.758778 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6298bd55-35da-4c79-ba5f-f6dc47cf416f admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:45.759001 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b679c9-ea82-4c31-81e8-11e4ff1979fc None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:45.761952 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-6298bd55-35da-4c79-ba5f-f6dc47cf416f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:45.797971 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b679c9-ea82-4c31-81e8-11e4ff1979fc None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:45.848286 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6298bd55-35da-4c79-ba5f-f6dc47cf416f admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:45.853862 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197e93e4-b8ee-4e08-b70a-d896b52ddbef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:45.854516 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197e93e4-b8ee-4e08-b70a-d896b52ddbef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:45.854794 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197e93e4-b8ee-4e08-b70a-d896b52ddbef admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:45.855202 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197e93e4-b8ee-4e08-b70a-d896b52ddbef admin admin] PATH_INFO: `/v3/projects/876a7d0ff0ec420d98d2f5711cbc4548` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:45.856357 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197e93e4-b8ee-4e08-b70a-d896b52ddbef admin admin] RBAC: Authorizing `identity:delete_project(project_id=876a7d0ff0ec420d98d2f5711cbc4548)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:45.858386 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197e93e4-b8ee-4e08-b70a-d896b52ddbef admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:45.879008 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc782f1-90ee-481e-bb3f-ae373e633c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:45.879425 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc782f1-90ee-481e-bb3f-ae373e633c48 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:45.879703 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc782f1-90ee-481e-bb3f-ae373e633c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:45.879974 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc782f1-90ee-481e-bb3f-ae373e633c48 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:45.880612 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc782f1-90ee-481e-bb3f-ae373e633c48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:45.882599 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc782f1-90ee-481e-bb3f-ae373e633c48 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:45.906162 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1012/2021] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:17:45 2022] POST /identity/v3/users => generated 292 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:17:45.910222 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc782f1-90ee-481e-bb3f-ae373e633c48 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:45.915896 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-197e93e4-b8ee-4e08-b70a-d896b52ddbef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '876a7d0ff0ec420d98d2f5711cbc4548', 'request_id': 'req-197e93e4-b8ee-4e08-b70a-d896b52ddbef'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:45.921007 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc782f1-90ee-481e-bb3f-ae373e633c48 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:45.925088 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-197e93e4-b8ee-4e08-b70a-d896b52ddbef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '876a7d0ff0ec420d98d2f5711cbc4548', 'request_id': 'req-197e93e4-b8ee-4e08-b70a-d896b52ddbef'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:45.938166 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-197e93e4-b8ee-4e08-b70a-d896b52ddbef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '876a7d0ff0ec420d98d2f5711cbc4548', 'request_id': 'req-197e93e4-b8ee-4e08-b70a-d896b52ddbef'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:45.951508 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:45.960721 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-197e93e4-b8ee-4e08-b70a-d896b52ddbef admin admin] The token cache is being invalidate because project 876a7d0ff0ec420d98d2f5711cbc4548 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:17:45.961106 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-197e93e4-b8ee-4e08-b70a-d896b52ddbef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:45.962301 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-197e93e4-b8ee-4e08-b70a-d896b52ddbef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:45.965090 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1010/2022] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:17:45 2022] DELETE /identity/v3/projects/876a7d0ff0ec420d98d2f5711cbc4548 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:45.971691 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28a2cbb-6f09-4dcf-9095-301a42094f22 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:45.972228 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28a2cbb-6f09-4dcf-9095-301a42094f22 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:45.972979 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28a2cbb-6f09-4dcf-9095-301a42094f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:45.985440 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:45.998965 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-a28a2cbb-6f09-4dcf-9095-301a42094f22 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:46.000275 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a28a2cbb-6f09-4dcf-9095-301a42094f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:46.048533 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61095c7-3976-4659-9be0-6600557097e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:46.050870 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61095c7-3976-4659-9be0-6600557097e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:46.051319 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61095c7-3976-4659-9be0-6600557097e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:46.051896 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61095c7-3976-4659-9be0-6600557097e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:46.053226 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61095c7-3976-4659-9be0-6600557097e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:46.055419 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61095c7-3976-4659-9be0-6600557097e1 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:46.079342 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1013/2023] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:17:45 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 16:17:46.083918 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61095c7-3976-4659-9be0-6600557097e1 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:46.086094 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f61095c7-3976-4659-9be0-6600557097e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:46.102568 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61095c7-3976-4659-9be0-6600557097e1 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:46.107554 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059fb21e-bca6-4c2f-b081-12d1be6c1ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:46.108178 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059fb21e-bca6-4c2f-b081-12d1be6c1ea8 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:46.108601 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059fb21e-bca6-4c2f-b081-12d1be6c1ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:46.108963 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059fb21e-bca6-4c2f-b081-12d1be6c1ea8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:46.109620 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059fb21e-bca6-4c2f-b081-12d1be6c1ea8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:46.111395 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059fb21e-bca6-4c2f-b081-12d1be6c1ea8 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:46.112919 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1011/2024] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:17:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:17:46.120579 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0075e5-b62b-4b2b-8133-985d4f2d73e3 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:46.123894 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0075e5-b62b-4b2b-8133-985d4f2d73e3 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:46.124293 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0075e5-b62b-4b2b-8133-985d4f2d73e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:46.124903 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1014/2025] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:17:46 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 16:17:46.129576 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059fb21e-bca6-4c2f-b081-12d1be6c1ea8 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:46.141649 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059fb21e-bca6-4c2f-b081-12d1be6c1ea8 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:46.147450 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d1d1a1-f698-4faf-b903-ad13c84981be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:46.148101 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d1d1a1-f698-4faf-b903-ad13c84981be admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:46.148429 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d1d1a1-f698-4faf-b903-ad13c84981be admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:46.148700 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d1d1a1-f698-4faf-b903-ad13c84981be admin admin] PATH_INFO: `/v3/projects/7320935cc63049dea292fefa5e6a5c91/users/390ca7ac2b9e4855aad3d1ed9ebd9e5e/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:46.158216 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-ac0075e5-b62b-4b2b-8133-985d4f2d73e3 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:46.160097 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ac0075e5-b62b-4b2b-8133-985d4f2d73e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:46.167057 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d1d1a1-f698-4faf-b903-ad13c84981be admin admin] RBAC: Authorizing `identity:create_grant(project_id=7320935cc63049dea292fefa5e6a5c91, user_id=390ca7ac2b9e4855aad3d1ed9ebd9e5e, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:46.169080 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1012/2026] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:17:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:46.170317 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d1d1a1-f698-4faf-b903-ad13c84981be admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:46.173147 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0075e5-b62b-4b2b-8133-985d4f2d73e3 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:46.185277 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0075e5-b62b-4b2b-8133-985d4f2d73e3 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:46.197905 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1015/2027] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:17:46 2022] PUT /identity/v3/projects/7320935cc63049dea292fefa5e6a5c91/users/390ca7ac2b9e4855aad3d1ed9ebd9e5e/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:46.209804 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bede31-a27f-4659-b3e7-2a7ae50b8a59 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:46.210199 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bede31-a27f-4659-b3e7-2a7ae50b8a59 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:46.210303 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bede31-a27f-4659-b3e7-2a7ae50b8a59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:46.282965 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-44bede31-a27f-4659-b3e7-2a7ae50b8a59 None None] MFA Rules not processed for user `390ca7ac2b9e4855aad3d1ed9ebd9e5e`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:46.284525 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-44bede31-a27f-4659-b3e7-2a7ae50b8a59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:46.308971 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31837e4a-7f19-48c3-9d1a-812fc88447d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:46.310593 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31837e4a-7f19-48c3-9d1a-812fc88447d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:46.311291 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31837e4a-7f19-48c3-9d1a-812fc88447d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:46.312319 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31837e4a-7f19-48c3-9d1a-812fc88447d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:46.315364 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31837e4a-7f19-48c3-9d1a-812fc88447d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:17:46.316491 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31837e4a-7f19-48c3-9d1a-812fc88447d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:46.319694 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31837e4a-7f19-48c3-9d1a-812fc88447d0 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:46.366562 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1013/2028] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 6 16:17:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:17:46.366562 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31837e4a-7f19-48c3-9d1a-812fc88447d0 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:46.370574 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-31837e4a-7f19-48c3-9d1a-812fc88447d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:46.390906 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31837e4a-7f19-48c3-9d1a-812fc88447d0 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:46.392696 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1016/2029] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:17:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:17:46.393876 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bede31-a27f-4659-b3e7-2a7ae50b8a59 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:46.397146 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98e4e73-9a34-46c5-a512-9fad41a82d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:46.398251 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98e4e73-9a34-46c5-a512-9fad41a82d55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:46.398738 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98e4e73-9a34-46c5-a512-9fad41a82d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:46.399249 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98e4e73-9a34-46c5-a512-9fad41a82d55 admin admin] PATH_INFO: `/v3/users/2a575e0f779a485e9918db75f1c7da7f` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:46.400473 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98e4e73-9a34-46c5-a512-9fad41a82d55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a575e0f779a485e9918db75f1c7da7f)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:46.402976 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98e4e73-9a34-46c5-a512-9fad41a82d55 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:46.405688 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bede31-a27f-4659-b3e7-2a7ae50b8a59 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:46.410765 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17639ff-eeb8-439a-a59e-ea255d252d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:46.411424 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17639ff-eeb8-439a-a59e-ea255d252d46 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:46.411853 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17639ff-eeb8-439a-a59e-ea255d252d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:46.412133 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17639ff-eeb8-439a-a59e-ea255d252d46 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:46.412793 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17639ff-eeb8-439a-a59e-ea255d252d46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:46.414829 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17639ff-eeb8-439a-a59e-ea255d252d46 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:46.447626 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1017/2030] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:17:46 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:17:46.449065 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17639ff-eeb8-439a-a59e-ea255d252d46 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:46.479165 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a98e4e73-9a34-46c5-a512-9fad41a82d55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a575e0f779a485e9918db75f1c7da7f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:46.480144 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a98e4e73-9a34-46c5-a512-9fad41a82d55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a575e0f779a485e9918db75f1c7da7f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:46.509378 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a98e4e73-9a34-46c5-a512-9fad41a82d55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a575e0f779a485e9918db75f1c7da7f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:46.532936 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a98e4e73-9a34-46c5-a512-9fad41a82d55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a575e0f779a485e9918db75f1c7da7f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:46.545163 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a98e4e73-9a34-46c5-a512-9fad41a82d55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a575e0f779a485e9918db75f1c7da7f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:46.547379 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:46.552363 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17639ff-eeb8-439a-a59e-ea255d252d46 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:46.555772 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1014/2031] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:17:46 2022] DELETE /identity/v3/users/2a575e0f779a485e9918db75f1c7da7f => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:46.556695 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98e4e73-9a34-46c5-a512-9fad41a82d55 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:46.559606 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a98e4e73-9a34-46c5-a512-9fad41a82d55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:46.560261 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc73cd69-dccc-4286-939c-3cf71e949200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:46.560960 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc73cd69-dccc-4286-939c-3cf71e949200 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:46.561798 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc73cd69-dccc-4286-939c-3cf71e949200 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:46.562154 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc73cd69-dccc-4286-939c-3cf71e949200 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:46.562806 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc73cd69-dccc-4286-939c-3cf71e949200 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:46.564925 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc73cd69-dccc-4286-939c-3cf71e949200 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:46.582692 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:46.602985 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1018/2032] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:17:46 2022] POST /identity/v3/projects => generated 383 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:17:46.602985 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0638681e-f813-4d6c-b555-d62b3c331a7a None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:46.602985 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0638681e-f813-4d6c-b555-d62b3c331a7a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:46.602985 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0638681e-f813-4d6c-b555-d62b3c331a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:46.621437 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-0638681e-f813-4d6c-b555-d62b3c331a7a None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:46.622654 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0638681e-f813-4d6c-b555-d62b3c331a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:46.640701 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98e4e73-9a34-46c5-a512-9fad41a82d55 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:46.690048 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1019/2033] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:17:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:46.690968 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0638681e-f813-4d6c-b555-d62b3c331a7a None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:46.704034 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0638681e-f813-4d6c-b555-d62b3c331a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:46.716568 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0638681e-f813-4d6c-b555-d62b3c331a7a None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:46.718923 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb9d8c4-d31f-4576-b3bf-eb28a73bbb0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:46.719688 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb9d8c4-d31f-4576-b3bf-eb28a73bbb0d admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:46.720032 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb9d8c4-d31f-4576-b3bf-eb28a73bbb0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:46.720372 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb9d8c4-d31f-4576-b3bf-eb28a73bbb0d admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:46.721231 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb9d8c4-d31f-4576-b3bf-eb28a73bbb0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:46.723542 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb9d8c4-d31f-4576-b3bf-eb28a73bbb0d admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:46.724293 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276d6e44-2d0b-41b8-9686-be3d9867dc82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:46.725371 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d6e44-2d0b-41b8-9686-be3d9867dc82 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:46.725779 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d6e44-2d0b-41b8-9686-be3d9867dc82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:46.726223 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d6e44-2d0b-41b8-9686-be3d9867dc82 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:46.727275 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276d6e44-2d0b-41b8-9686-be3d9867dc82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:46.729977 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276d6e44-2d0b-41b8-9686-be3d9867dc82 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:46.753203 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1015/2034] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:17:46 2022] POST /identity/v3/users => generated 293 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:17:46.754983 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1020/2035] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:17:46 2022] POST /identity/v3/users => generated 298 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:17:46.757609 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276d6e44-2d0b-41b8-9686-be3d9867dc82 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:46.757742 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb9d8c4-d31f-4576-b3bf-eb28a73bbb0d admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:46.760011 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-6bb9d8c4-d31f-4576-b3bf-eb28a73bbb0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:46.768443 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276d6e44-2d0b-41b8-9686-be3d9867dc82 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:46.773727 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905e5d54-ff3e-41e0-b4ce-3e7b5a1f43d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:46.774358 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e5d54-ff3e-41e0-b4ce-3e7b5a1f43d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:46.774639 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e5d54-ff3e-41e0-b4ce-3e7b5a1f43d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:46.774909 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e5d54-ff3e-41e0-b4ce-3e7b5a1f43d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:46.775563 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905e5d54-ff3e-41e0-b4ce-3e7b5a1f43d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:46.777261 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905e5d54-ff3e-41e0-b4ce-3e7b5a1f43d1 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:46.792702 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1021/2036] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:17:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:17:46.793728 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905e5d54-ff3e-41e0-b4ce-3e7b5a1f43d1 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:46.811216 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905e5d54-ff3e-41e0-b4ce-3e7b5a1f43d1 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:46.817187 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6c2c0b-8dbc-43c8-8323-e4941b7b8e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:46.817835 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c2c0b-8dbc-43c8-8323-e4941b7b8e55 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:46.818122 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c2c0b-8dbc-43c8-8323-e4941b7b8e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:46.818473 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c2c0b-8dbc-43c8-8323-e4941b7b8e55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:46.819095 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6c2c0b-8dbc-43c8-8323-e4941b7b8e55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:46.820866 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6c2c0b-8dbc-43c8-8323-e4941b7b8e55 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:46.849007 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1022/2037] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:17:46 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:17:46.849007 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6c2c0b-8dbc-43c8-8323-e4941b7b8e55 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:46.849007 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6c2c0b-8dbc-43c8-8323-e4941b7b8e55 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:46.851683 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb9d8c4-d31f-4576-b3bf-eb28a73bbb0d admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:46.853571 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2c5a6b-cb41-4201-8c89-40a099eb89b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:46.854206 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2c5a6b-cb41-4201-8c89-40a099eb89b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:46.854483 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2c5a6b-cb41-4201-8c89-40a099eb89b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:46.854751 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2c5a6b-cb41-4201-8c89-40a099eb89b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:46.855407 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2c5a6b-cb41-4201-8c89-40a099eb89b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:46.855691 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a85979-a48f-47cf-b765-76c013f0aacb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:46.856331 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a85979-a48f-47cf-b765-76c013f0aacb service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:46.856612 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a85979-a48f-47cf-b765-76c013f0aacb service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:46.856925 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a85979-a48f-47cf-b765-76c013f0aacb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:46.857273 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2c5a6b-cb41-4201-8c89-40a099eb89b1 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:46.867356 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a85979-a48f-47cf-b765-76c013f0aacb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:46.868878 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a85979-a48f-47cf-b765-76c013f0aacb service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:46.869263 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1023/2038] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:17:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:17:46.873576 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2c5a6b-cb41-4201-8c89-40a099eb89b1 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:46.880124 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1016/2039] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:17:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:17:46.881040 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a85979-a48f-47cf-b765-76c013f0aacb service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:46.884284 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2c5a6b-cb41-4201-8c89-40a099eb89b1 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:46.889750 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217f11ed-e0e7-4360-ae29-e515bf2252fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:46.890592 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217f11ed-e0e7-4360-ae29-e515bf2252fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:46.890939 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217f11ed-e0e7-4360-ae29-e515bf2252fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:46.891638 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217f11ed-e0e7-4360-ae29-e515bf2252fb admin admin] PATH_INFO: `/v3/projects/4cb1adafc10f46dd816f346effdbad68/users/4fb91f49e05e4e4bab85a10f8a4e9f48/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:46.891741 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a85979-a48f-47cf-b765-76c013f0aacb service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:46.896927 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e403846-b8e8-426a-95de-6a4352c3d4e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:46.897846 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e403846-b8e8-426a-95de-6a4352c3d4e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:46.897957 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e403846-b8e8-426a-95de-6a4352c3d4e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:46.898191 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e403846-b8e8-426a-95de-6a4352c3d4e2 admin admin] PATH_INFO: `/v3/projects/04bbe101d7ac40ab995da2159f3105cd/users/bbe4c1ecf08f4dd5a005f883c16c6a62/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:46.911816 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217f11ed-e0e7-4360-ae29-e515bf2252fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cb1adafc10f46dd816f346effdbad68, user_id=4fb91f49e05e4e4bab85a10f8a4e9f48, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:46.914810 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217f11ed-e0e7-4360-ae29-e515bf2252fb admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:46.918062 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e403846-b8e8-426a-95de-6a4352c3d4e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04bbe101d7ac40ab995da2159f3105cd, user_id=bbe4c1ecf08f4dd5a005f883c16c6a62, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:46.921129 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e403846-b8e8-426a-95de-6a4352c3d4e2 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:46.942791 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1024/2040] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:17:46 2022] PUT /identity/v3/projects/4cb1adafc10f46dd816f346effdbad68/users/4fb91f49e05e4e4bab85a10f8a4e9f48/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:46.950235 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217f11ed-e0e7-4360-ae29-e515bf2252fb admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:46.954995 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1017/2041] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:17:46 2022] PUT /identity/v3/projects/04bbe101d7ac40ab995da2159f3105cd/users/bbe4c1ecf08f4dd5a005f883c16c6a62/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:46.961540 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae389ea7-e00c-4474-9580-b94bfe2191bc None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:46.962049 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae389ea7-e00c-4474-9580-b94bfe2191bc None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:46.962514 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae389ea7-e00c-4474-9580-b94bfe2191bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:46.997890 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-ae389ea7-e00c-4474-9580-b94bfe2191bc None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:46.999411 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ae389ea7-e00c-4474-9580-b94bfe2191bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:47.012402 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:47.063762 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217f11ed-e0e7-4360-ae29-e515bf2252fb admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:47.101669 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:47.115222 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1018/2042] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:17:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:47.129711 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae389ea7-e00c-4474-9580-b94bfe2191bc None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:47.164922 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370cb003-5abb-4343-9149-12b8f8be3f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:47.166047 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370cb003-5abb-4343-9149-12b8f8be3f19 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:47.166695 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370cb003-5abb-4343-9149-12b8f8be3f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:47.167169 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370cb003-5abb-4343-9149-12b8f8be3f19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:47.168419 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370cb003-5abb-4343-9149-12b8f8be3f19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:47.170444 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370cb003-5abb-4343-9149-12b8f8be3f19 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:47.187033 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1025/2043] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:17:46 2022] GET /identity/v3/roles => generated 1420 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 16:17:47.191482 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370cb003-5abb-4343-9149-12b8f8be3f19 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:47.206617 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370cb003-5abb-4343-9149-12b8f8be3f19 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:47.210478 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373f5958-7a11-4110-94a0-f96be4747c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:47.211359 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373f5958-7a11-4110-94a0-f96be4747c28 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:47.211774 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373f5958-7a11-4110-94a0-f96be4747c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:47.212052 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373f5958-7a11-4110-94a0-f96be4747c28 admin admin] PATH_INFO: `/v3/projects/4cb1adafc10f46dd816f346effdbad68/users/4fb91f49e05e4e4bab85a10f8a4e9f48/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:47.214496 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373f5958-7a11-4110-94a0-f96be4747c28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cb1adafc10f46dd816f346effdbad68, user_id=4fb91f49e05e4e4bab85a10f8a4e9f48, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:47.217681 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373f5958-7a11-4110-94a0-f96be4747c28 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:47.244551 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae389ea7-e00c-4474-9580-b94bfe2191bc None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:47.251368 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1026/2044] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:17:47 2022] PUT /identity/v3/projects/4cb1adafc10f46dd816f346effdbad68/users/4fb91f49e05e4e4bab85a10f8a4e9f48/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:47.263423 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164fe8d6-05e3-4984-93d9-928d9d0eb32e None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:47.263705 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164fe8d6-05e3-4984-93d9-928d9d0eb32e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:47.263976 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164fe8d6-05e3-4984-93d9-928d9d0eb32e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:47.327688 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-164fe8d6-05e3-4984-93d9-928d9d0eb32e None None] MFA Rules not processed for user `4fb91f49e05e4e4bab85a10f8a4e9f48`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:47.329159 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-164fe8d6-05e3-4984-93d9-928d9d0eb32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:47.343107 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01100f8b-a747-402d-a446-4bf9e2a751a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:47.343798 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01100f8b-a747-402d-a446-4bf9e2a751a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:47.344290 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01100f8b-a747-402d-a446-4bf9e2a751a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:47.344385 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01100f8b-a747-402d-a446-4bf9e2a751a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:47.448449 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1027/2045] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:17:47 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:47.450328 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164fe8d6-05e3-4984-93d9-928d9d0eb32e None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:47.451887 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-164fe8d6-05e3-4984-93d9-928d9d0eb32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:47.461316 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01100f8b-a747-402d-a446-4bf9e2a751a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:47.463659 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01100f8b-a747-402d-a446-4bf9e2a751a6 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:47.469641 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164fe8d6-05e3-4984-93d9-928d9d0eb32e None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:47.475847 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1019/2046] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:17:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:17:47.476214 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7892f0b1-4dce-458d-9676-0901383fa211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:47.481039 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7892f0b1-4dce-458d-9676-0901383fa211 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:47.481434 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7892f0b1-4dce-458d-9676-0901383fa211 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:47.481731 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7892f0b1-4dce-458d-9676-0901383fa211 admin admin] PATH_INFO: `/v3/projects/61cede192fdd4fbca3e14fc7c17a9ccd` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:47.483143 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7892f0b1-4dce-458d-9676-0901383fa211 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61cede192fdd4fbca3e14fc7c17a9ccd)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:47.485345 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7892f0b1-4dce-458d-9676-0901383fa211 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:47.486499 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01100f8b-a747-402d-a446-4bf9e2a751a6 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:47.490055 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-01100f8b-a747-402d-a446-4bf9e2a751a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:47.527411 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:47.573259 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7892f0b1-4dce-458d-9676-0901383fa211 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61cede192fdd4fbca3e14fc7c17a9ccd', 'request_id': 'req-7892f0b1-4dce-458d-9676-0901383fa211'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:47.580116 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01100f8b-a747-402d-a446-4bf9e2a751a6 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:47.587079 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7892f0b1-4dce-458d-9676-0901383fa211 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61cede192fdd4fbca3e14fc7c17a9ccd', 'request_id': 'req-7892f0b1-4dce-458d-9676-0901383fa211'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:47.604985 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7892f0b1-4dce-458d-9676-0901383fa211 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61cede192fdd4fbca3e14fc7c17a9ccd', 'request_id': 'req-7892f0b1-4dce-458d-9676-0901383fa211'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:47.624001 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7892f0b1-4dce-458d-9676-0901383fa211 admin admin] The token cache is being invalidate because project 61cede192fdd4fbca3e14fc7c17a9ccd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:17:47.624373 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7892f0b1-4dce-458d-9676-0901383fa211 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:47.625233 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7892f0b1-4dce-458d-9676-0901383fa211 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:47.631288 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1028/2047] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:17:47 2022] DELETE /identity/v3/projects/61cede192fdd4fbca3e14fc7c17a9ccd => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:47.632399 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7892f0b1-4dce-458d-9676-0901383fa211 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:47.635245 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-7892f0b1-4dce-458d-9676-0901383fa211 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:47.681109 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:47.711341 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85704c3-1077-4721-8491-8e28a8a3c4cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:47.712025 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85704c3-1077-4721-8491-8e28a8a3c4cc service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:47.712298 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85704c3-1077-4721-8491-8e28a8a3c4cc service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:47.712567 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85704c3-1077-4721-8491-8e28a8a3c4cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:47.714549 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d85704c3-1077-4721-8491-8e28a8a3c4cc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:47.754830 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7892f0b1-4dce-458d-9676-0901383fa211 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:47.770894 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bf1c6b-654c-4739-8e82-c2fd9997b0fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:47.771585 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bf1c6b-654c-4739-8e82-c2fd9997b0fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:47.771838 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bf1c6b-654c-4739-8e82-c2fd9997b0fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:47.772253 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bf1c6b-654c-4739-8e82-c2fd9997b0fb admin admin] PATH_INFO: `/v3/users/8954a22e9f324f52b432801f19215e1d` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:47.775281 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bf1c6b-654c-4739-8e82-c2fd9997b0fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=8954a22e9f324f52b432801f19215e1d)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:47.777348 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bf1c6b-654c-4739-8e82-c2fd9997b0fb admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:47.833890 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85704c3-1077-4721-8491-8e28a8a3c4cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:47.836052 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85704c3-1077-4721-8491-8e28a8a3c4cc service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:47.850745 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1020/2048] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:17:47 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:17:47.909582 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-57bf1c6b-654c-4739-8e82-c2fd9997b0fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8954a22e9f324f52b432801f19215e1d'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:47.911651 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-57bf1c6b-654c-4739-8e82-c2fd9997b0fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8954a22e9f324f52b432801f19215e1d'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:47.931660 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-57bf1c6b-654c-4739-8e82-c2fd9997b0fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8954a22e9f324f52b432801f19215e1d'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:47.957046 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-57bf1c6b-654c-4739-8e82-c2fd9997b0fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8954a22e9f324f52b432801f19215e1d'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:47.958487 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-57bf1c6b-654c-4739-8e82-c2fd9997b0fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8954a22e9f324f52b432801f19215e1d'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:47.985855 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1029/2049] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:17:47 2022] DELETE /identity/v3/users/8954a22e9f324f52b432801f19215e1d => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:48.472319 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85704c3-1077-4721-8491-8e28a8a3c4cc service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:48.485012 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85704c3-1077-4721-8491-8e28a8a3c4cc service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:48.553063 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3300e04d-c49a-4050-9180-2f76498524d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:48.553736 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3300e04d-c49a-4050-9180-2f76498524d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:48.554095 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3300e04d-c49a-4050-9180-2f76498524d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:48.554365 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3300e04d-c49a-4050-9180-2f76498524d4 admin admin] PATH_INFO: `/v3/projects/510f6230c5d545338f64da15c5f2ea28` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:48.555806 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3300e04d-c49a-4050-9180-2f76498524d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=510f6230c5d545338f64da15c5f2ea28)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:48.557803 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3300e04d-c49a-4050-9180-2f76498524d4 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:48.597868 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:48.617054 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3300e04d-c49a-4050-9180-2f76498524d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '510f6230c5d545338f64da15c5f2ea28', 'request_id': 'req-3300e04d-c49a-4050-9180-2f76498524d4'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:48.627810 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3300e04d-c49a-4050-9180-2f76498524d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '510f6230c5d545338f64da15c5f2ea28', 'request_id': 'req-3300e04d-c49a-4050-9180-2f76498524d4'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:48.639155 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3300e04d-c49a-4050-9180-2f76498524d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '510f6230c5d545338f64da15c5f2ea28', 'request_id': 'req-3300e04d-c49a-4050-9180-2f76498524d4'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:48.654379 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3300e04d-c49a-4050-9180-2f76498524d4 admin admin] The token cache is being invalidate because project 510f6230c5d545338f64da15c5f2ea28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:17:48.654669 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3300e04d-c49a-4050-9180-2f76498524d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:48.655367 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3300e04d-c49a-4050-9180-2f76498524d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:48.657038 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1021/2050] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:17:48 2022] DELETE /identity/v3/projects/510f6230c5d545338f64da15c5f2ea28 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:48.663585 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bf1c6b-654c-4739-8e82-c2fd9997b0fb admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:48.665561 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-57bf1c6b-654c-4739-8e82-c2fd9997b0fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:48.680869 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bf1c6b-654c-4739-8e82-c2fd9997b0fb admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:48.817183 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae7c9b2-43f0-4a6f-b145-a060826f5e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:48.817922 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae7c9b2-43f0-4a6f-b145-a060826f5e6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:48.818224 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae7c9b2-43f0-4a6f-b145-a060826f5e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:48.818699 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae7c9b2-43f0-4a6f-b145-a060826f5e6c admin admin] PATH_INFO: `/v3/users/d77803c6aa7b4a1a8ece9714d8d5e0b4` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:48.820524 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae7c9b2-43f0-4a6f-b145-a060826f5e6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d77803c6aa7b4a1a8ece9714d8d5e0b4)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:48.822220 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae7c9b2-43f0-4a6f-b145-a060826f5e6c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:48.912056 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:48.956553 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2ae7c9b2-43f0-4a6f-b145-a060826f5e6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd77803c6aa7b4a1a8ece9714d8d5e0b4'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:48.958616 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2ae7c9b2-43f0-4a6f-b145-a060826f5e6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd77803c6aa7b4a1a8ece9714d8d5e0b4'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:48.996038 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2ae7c9b2-43f0-4a6f-b145-a060826f5e6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd77803c6aa7b4a1a8ece9714d8d5e0b4'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:49.052546 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2ae7c9b2-43f0-4a6f-b145-a060826f5e6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd77803c6aa7b4a1a8ece9714d8d5e0b4'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:49.057391 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2ae7c9b2-43f0-4a6f-b145-a060826f5e6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd77803c6aa7b4a1a8ece9714d8d5e0b4'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:49.093052 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1030/2051] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:17:48 2022] DELETE /identity/v3/users/d77803c6aa7b4a1a8ece9714d8d5e0b4 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:49.733808 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3300e04d-c49a-4050-9180-2f76498524d4 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:49.735482 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-3300e04d-c49a-4050-9180-2f76498524d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:49.816997 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3300e04d-c49a-4050-9180-2f76498524d4 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:49.884021 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fb5977-f556-47eb-bf47-a32667d19c07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:49.884630 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fb5977-f556-47eb-bf47-a32667d19c07 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:49.884946 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fb5977-f556-47eb-bf47-a32667d19c07 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:49.885223 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fb5977-f556-47eb-bf47-a32667d19c07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:49.887416 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f4fb5977-f556-47eb-bf47-a32667d19c07 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:49.986006 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fb5977-f556-47eb-bf47-a32667d19c07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:49.988561 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fb5977-f556-47eb-bf47-a32667d19c07 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:50.007741 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1022/2052] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:17:49 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:17:50.025858 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae7c9b2-43f0-4a6f-b145-a060826f5e6c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:50.027825 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-2ae7c9b2-43f0-4a6f-b145-a060826f5e6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:50.156118 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae7c9b2-43f0-4a6f-b145-a060826f5e6c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:50.173062 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b79d159-da46-430f-bc93-79cca2712e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:50.173823 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b79d159-da46-430f-bc93-79cca2712e47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:50.174104 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b79d159-da46-430f-bc93-79cca2712e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:50.174380 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b79d159-da46-430f-bc93-79cca2712e47 admin admin] PATH_INFO: `/v3/projects/646c9afc62954c9bad1b81c6920a7663` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:50.176490 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b79d159-da46-430f-bc93-79cca2712e47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=646c9afc62954c9bad1b81c6920a7663)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:50.185414 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b79d159-da46-430f-bc93-79cca2712e47 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:50.304931 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5b79d159-da46-430f-bc93-79cca2712e47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '646c9afc62954c9bad1b81c6920a7663', 'request_id': 'req-5b79d159-da46-430f-bc93-79cca2712e47'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:50.319912 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5b79d159-da46-430f-bc93-79cca2712e47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '646c9afc62954c9bad1b81c6920a7663', 'request_id': 'req-5b79d159-da46-430f-bc93-79cca2712e47'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:50.340967 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5b79d159-da46-430f-bc93-79cca2712e47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '646c9afc62954c9bad1b81c6920a7663', 'request_id': 'req-5b79d159-da46-430f-bc93-79cca2712e47'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:50.387538 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5b79d159-da46-430f-bc93-79cca2712e47 admin admin] The token cache is being invalidate because project 646c9afc62954c9bad1b81c6920a7663 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:17:50.388044 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5b79d159-da46-430f-bc93-79cca2712e47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:50.389274 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5b79d159-da46-430f-bc93-79cca2712e47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:50.393547 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1031/2053] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:17:50 2022] DELETE /identity/v3/projects/646c9afc62954c9bad1b81c6920a7663 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:50.400162 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd45789-aa6c-4ac8-8b64-d1bbd05f2250 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:50.401036 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd45789-aa6c-4ac8-8b64-d1bbd05f2250 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:50.401501 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd45789-aa6c-4ac8-8b64-d1bbd05f2250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:50.496133 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-6fd45789-aa6c-4ac8-8b64-d1bbd05f2250 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:50.500903 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-6fd45789-aa6c-4ac8-8b64-d1bbd05f2250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:50.536280 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:50.634921 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1023/2054] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:17:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:50.642918 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b532dd7-7cf1-46a5-a6a6-77bb18888754 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:50.643823 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b532dd7-7cf1-46a5-a6a6-77bb18888754 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:50.644214 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b532dd7-7cf1-46a5-a6a6-77bb18888754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:50.674727 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-2b532dd7-7cf1-46a5-a6a6-77bb18888754 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:50.676184 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-2b532dd7-7cf1-46a5-a6a6-77bb18888754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:50.688332 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1032/2055] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:17:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:50.693230 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd45789-aa6c-4ac8-8b64-d1bbd05f2250 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:50.721088 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd45789-aa6c-4ac8-8b64-d1bbd05f2250 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:50.732484 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c061ffd-63bc-4fe1-99c6-be101f27563f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:50.733808 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c061ffd-63bc-4fe1-99c6-be101f27563f admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:50.734443 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c061ffd-63bc-4fe1-99c6-be101f27563f admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:50.734972 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c061ffd-63bc-4fe1-99c6-be101f27563f admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:50.736991 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c061ffd-63bc-4fe1-99c6-be101f27563f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:17:50.737637 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c061ffd-63bc-4fe1-99c6-be101f27563f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:50.742893 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c061ffd-63bc-4fe1-99c6-be101f27563f admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:50.789974 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1024/2056] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 6 16:17:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:17:50.792094 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b532dd7-7cf1-46a5-a6a6-77bb18888754 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:50.806285 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b532dd7-7cf1-46a5-a6a6-77bb18888754 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:50.813625 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31edd6e7-7502-4d22-a263-a9e64e6f7bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:50.814847 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31edd6e7-7502-4d22-a263-a9e64e6f7bfe admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:50.815115 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31edd6e7-7502-4d22-a263-a9e64e6f7bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:50.815638 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31edd6e7-7502-4d22-a263-a9e64e6f7bfe admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:50.816852 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31edd6e7-7502-4d22-a263-a9e64e6f7bfe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:50.819203 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31edd6e7-7502-4d22-a263-a9e64e6f7bfe admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:50.856815 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1033/2057] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:17:50 2022] POST /identity/v3/projects => generated 371 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:17:50.860092 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c061ffd-63bc-4fe1-99c6-be101f27563f admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:50.900815 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c061ffd-63bc-4fe1-99c6-be101f27563f admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:51.014373 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd02c8d8-4327-4c6e-be7f-3b58fa6cd44c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:51.015296 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd02c8d8-4327-4c6e-be7f-3b58fa6cd44c admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:51.015779 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd02c8d8-4327-4c6e-be7f-3b58fa6cd44c admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:51.016194 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd02c8d8-4327-4c6e-be7f-3b58fa6cd44c admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:51.017467 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd02c8d8-4327-4c6e-be7f-3b58fa6cd44c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:51.020303 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd02c8d8-4327-4c6e-be7f-3b58fa6cd44c admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:51.055166 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1025/2058] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:17:50 2022] POST /identity/v3/users => generated 286 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:17:51.059390 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31edd6e7-7502-4d22-a263-a9e64e6f7bfe admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:51.076042 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31edd6e7-7502-4d22-a263-a9e64e6f7bfe admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:51.082846 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857ffa70-9eeb-48db-8f8e-626474876abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:51.104891 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ffa70-9eeb-48db-8f8e-626474876abf admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:51.105387 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ffa70-9eeb-48db-8f8e-626474876abf admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:51.105814 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ffa70-9eeb-48db-8f8e-626474876abf admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:51.106665 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857ffa70-9eeb-48db-8f8e-626474876abf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:51.108477 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857ffa70-9eeb-48db-8f8e-626474876abf admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:51.130067 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1034/2059] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:17:51 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:17:51.130246 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd02c8d8-4327-4c6e-be7f-3b58fa6cd44c admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:51.141330 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd02c8d8-4327-4c6e-be7f-3b58fa6cd44c admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:51.148564 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857ffa70-9eeb-48db-8f8e-626474876abf admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:51.150640 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-857ffa70-9eeb-48db-8f8e-626474876abf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:51.150969 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a67542-7d64-4d39-bc00-a76469998fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:51.151786 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a67542-7d64-4d39-bc00-a76469998fc0 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:51.152081 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a67542-7d64-4d39-bc00-a76469998fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:51.152352 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a67542-7d64-4d39-bc00-a76469998fc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:51.153063 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a67542-7d64-4d39-bc00-a76469998fc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:51.154756 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a67542-7d64-4d39-bc00-a76469998fc0 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:51.167098 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:51.188543 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1026/2060] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:17:51 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 16:17:51.192993 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a67542-7d64-4d39-bc00-a76469998fc0 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:51.221472 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a67542-7d64-4d39-bc00-a76469998fc0 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:51.237935 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ee6855-2963-47b3-8c40-3f7887a60d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:51.239690 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ee6855-2963-47b3-8c40-3f7887a60d04 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:51.240119 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ee6855-2963-47b3-8c40-3f7887a60d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:51.242091 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ee6855-2963-47b3-8c40-3f7887a60d04 admin admin] PATH_INFO: `/v3/projects/4a2dcd76a5f24bd7ba164032cbe22d20/users/2921386cc370487da26a786c8198c94c/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:51.289917 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ee6855-2963-47b3-8c40-3f7887a60d04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a2dcd76a5f24bd7ba164032cbe22d20, user_id=2921386cc370487da26a786c8198c94c, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:51.293500 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ee6855-2963-47b3-8c40-3f7887a60d04 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:51.321393 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857ffa70-9eeb-48db-8f8e-626474876abf admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:51.324000 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1027/2061] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:17:51 2022] PUT /identity/v3/projects/4a2dcd76a5f24bd7ba164032cbe22d20/users/2921386cc370487da26a786c8198c94c/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:51.325894 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d28915-583a-4072-8ebb-d3e0fd79eb68 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:51.326305 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d28915-583a-4072-8ebb-d3e0fd79eb68 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:51.326663 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d28915-583a-4072-8ebb-d3e0fd79eb68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:51.381724 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-48d28915-583a-4072-8ebb-d3e0fd79eb68 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:51.382590 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-48d28915-583a-4072-8ebb-d3e0fd79eb68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:51.455178 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6ec36f-ab8d-4b49-9e33-da9b26b84c3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:51.456230 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6ec36f-ab8d-4b49-9e33-da9b26b84c3b service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:51.456866 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6ec36f-ab8d-4b49-9e33-da9b26b84c3b service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:51.457326 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6ec36f-ab8d-4b49-9e33-da9b26b84c3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:51.466192 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0f6ec36f-ab8d-4b49-9e33-da9b26b84c3b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:51.562789 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1028/2062] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:17:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:51.580438 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d28915-583a-4072-8ebb-d3e0fd79eb68 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:51.585371 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-48d28915-583a-4072-8ebb-d3e0fd79eb68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:51.625904 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:51.628390 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6ec36f-ab8d-4b49-9e33-da9b26b84c3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:51.630852 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6ec36f-ab8d-4b49-9e33-da9b26b84c3b service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:51.648351 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1035/2063] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:17:51 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:17:51.786700 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d28915-583a-4072-8ebb-d3e0fd79eb68 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:51.796583 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdf1c55-39cb-4e6e-8a6f-bc3acec54035 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:51.797249 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdf1c55-39cb-4e6e-8a6f-bc3acec54035 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:51.797515 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdf1c55-39cb-4e6e-8a6f-bc3acec54035 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:51.797770 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdf1c55-39cb-4e6e-8a6f-bc3acec54035 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:51.826701 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdf1c55-39cb-4e6e-8a6f-bc3acec54035 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:51.828853 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdf1c55-39cb-4e6e-8a6f-bc3acec54035 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:51.842690 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1029/2064] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:17:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:17:53.139843 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354d7605-2ab3-489e-a728-2bf1a57e52fa None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:53.141879 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354d7605-2ab3-489e-a728-2bf1a57e52fa None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:53.142403 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354d7605-2ab3-489e-a728-2bf1a57e52fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:53.223890 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-354d7605-2ab3-489e-a728-2bf1a57e52fa None None] MFA Rules not processed for user `2921386cc370487da26a786c8198c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:53.225141 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-354d7605-2ab3-489e-a728-2bf1a57e52fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:53.388267 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1036/2065] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:17:53 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:53.482299 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdf1c55-39cb-4e6e-8a6f-bc3acec54035 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:53.498520 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdf1c55-39cb-4e6e-8a6f-bc3acec54035 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:53.510781 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d212ac71-84d0-4c4e-b29a-3fbd2974842e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:53.513057 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d212ac71-84d0-4c4e-b29a-3fbd2974842e service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:53.514869 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d212ac71-84d0-4c4e-b29a-3fbd2974842e service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:53.515320 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d212ac71-84d0-4c4e-b29a-3fbd2974842e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:53.536545 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d212ac71-84d0-4c4e-b29a-3fbd2974842e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:53.539572 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d212ac71-84d0-4c4e-b29a-3fbd2974842e service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:53.564362 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1030/2066] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:17:53 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:17:55.558586 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854ba664-6182-41df-83ca-d2e34c75f08f None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:55.559063 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854ba664-6182-41df-83ca-d2e34c75f08f None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:55.559333 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854ba664-6182-41df-83ca-d2e34c75f08f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:55.567583 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:55.595603 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-854ba664-6182-41df-83ca-d2e34c75f08f None None] MFA Rules not processed for user `bbe4c1ecf08f4dd5a005f883c16c6a62`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:55.597066 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-854ba664-6182-41df-83ca-d2e34c75f08f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:55.724627 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d212ac71-84d0-4c4e-b29a-3fbd2974842e service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:55.730141 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d212ac71-84d0-4c4e-b29a-3fbd2974842e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:55.746148 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1037/2067] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:17:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:55.750486 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854ba664-6182-41df-83ca-d2e34c75f08f None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:55.751994 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-854ba664-6182-41df-83ca-d2e34c75f08f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:55.761958 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d212ac71-84d0-4c4e-b29a-3fbd2974842e service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:55.767541 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35e6456-29ef-442e-aae8-729261c4f744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:55.767834 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854ba664-6182-41df-83ca-d2e34c75f08f None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:55.768317 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35e6456-29ef-442e-aae8-729261c4f744 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:55.768595 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35e6456-29ef-442e-aae8-729261c4f744 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:55.768942 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35e6456-29ef-442e-aae8-729261c4f744 admin admin] PATH_INFO: `/v3/users/8978e7cc78ab4c559dfd3e22cde34569` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:55.770477 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35e6456-29ef-442e-aae8-729261c4f744 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8978e7cc78ab4c559dfd3e22cde34569)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:55.772608 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35e6456-29ef-442e-aae8-729261c4f744 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:55.774592 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03840ffa-616a-4b9d-af9f-0d244ac9e614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:55.775212 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03840ffa-616a-4b9d-af9f-0d244ac9e614 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:55.775673 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03840ffa-616a-4b9d-af9f-0d244ac9e614 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:55.776113 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03840ffa-616a-4b9d-af9f-0d244ac9e614 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:55.776630 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03840ffa-616a-4b9d-af9f-0d244ac9e614 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:55.778832 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03840ffa-616a-4b9d-af9f-0d244ac9e614 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:55.826727 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1038/2068] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:17:55 2022] POST /identity/v3/projects => generated 385 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:17:55.831128 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03840ffa-616a-4b9d-af9f-0d244ac9e614 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:55.853422 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a35e6456-29ef-442e-aae8-729261c4f744 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8978e7cc78ab4c559dfd3e22cde34569'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:55.854189 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a35e6456-29ef-442e-aae8-729261c4f744 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8978e7cc78ab4c559dfd3e22cde34569'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:55.865138 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a35e6456-29ef-442e-aae8-729261c4f744 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8978e7cc78ab4c559dfd3e22cde34569'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:55.895288 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a35e6456-29ef-442e-aae8-729261c4f744 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8978e7cc78ab4c559dfd3e22cde34569'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:55.903279 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a35e6456-29ef-442e-aae8-729261c4f744 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8978e7cc78ab4c559dfd3e22cde34569'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:55.915747 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1031/2069] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:17:55 2022] DELETE /identity/v3/users/8978e7cc78ab4c559dfd3e22cde34569 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:55.999424 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03840ffa-616a-4b9d-af9f-0d244ac9e614 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:56.082924 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b11b85-5206-4369-965c-f6f28b008838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:56.083710 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b11b85-5206-4369-965c-f6f28b008838 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:56.084069 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b11b85-5206-4369-965c-f6f28b008838 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:56.084456 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b11b85-5206-4369-965c-f6f28b008838 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:56.085301 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b11b85-5206-4369-965c-f6f28b008838 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:56.087867 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b11b85-5206-4369-965c-f6f28b008838 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:56.122228 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1039/2070] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:17:55 2022] POST /identity/v3/users => generated 299 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:17:56.126465 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35e6456-29ef-442e-aae8-729261c4f744 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:56.129611 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a35e6456-29ef-442e-aae8-729261c4f744 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:56.173237 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35e6456-29ef-442e-aae8-729261c4f744 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:56.178712 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2d7974-ad8c-4928-9849-5f0f4f567cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:56.179418 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2d7974-ad8c-4928-9849-5f0f4f567cee admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:56.179762 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2d7974-ad8c-4928-9849-5f0f4f567cee admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:56.180108 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2d7974-ad8c-4928-9849-5f0f4f567cee admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:56.180900 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2d7974-ad8c-4928-9849-5f0f4f567cee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:56.184923 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2d7974-ad8c-4928-9849-5f0f4f567cee admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:56.202296 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1032/2071] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:17:56 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:17:56.208161 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b11b85-5206-4369-965c-f6f28b008838 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:56.242651 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b11b85-5206-4369-965c-f6f28b008838 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:56.256007 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2d7974-ad8c-4928-9849-5f0f4f567cee admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:56.257924 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-4b2d7974-ad8c-4928-9849-5f0f4f567cee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:56.278565 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ff89b-9eeb-4e2d-94cc-22f0e328dd8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:56.279374 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ff89b-9eeb-4e2d-94cc-22f0e328dd8b admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:56.279746 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ff89b-9eeb-4e2d-94cc-22f0e328dd8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:56.280087 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ff89b-9eeb-4e2d-94cc-22f0e328dd8b admin admin] PATH_INFO: `/v3/projects/009f1033f43945cd8483afafd38f32b3/users/a280595c6ff4495ab9f4120b4bcc60c7/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:56.286573 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:56.305191 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4ff89b-9eeb-4e2d-94cc-22f0e328dd8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=009f1033f43945cd8483afafd38f32b3, user_id=a280595c6ff4495ab9f4120b4bcc60c7, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:56.308293 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2d7974-ad8c-4928-9849-5f0f4f567cee admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:56.308608 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4ff89b-9eeb-4e2d-94cc-22f0e328dd8b admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:56.317041 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6c0cd1-cdd1-4a42-a4bb-7890fcda053d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:56.318087 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c0cd1-cdd1-4a42-a4bb-7890fcda053d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:56.318602 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c0cd1-cdd1-4a42-a4bb-7890fcda053d admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:56.319428 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c0cd1-cdd1-4a42-a4bb-7890fcda053d admin admin] PATH_INFO: `/v3/projects/c66250df2e054415b517b2de8857bc87` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:56.320874 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6c0cd1-cdd1-4a42-a4bb-7890fcda053d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c66250df2e054415b517b2de8857bc87)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:56.324089 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6c0cd1-cdd1-4a42-a4bb-7890fcda053d admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:56.335711 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1040/2072] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:17:56 2022] PUT /identity/v3/projects/009f1033f43945cd8483afafd38f32b3/users/a280595c6ff4495ab9f4120b4bcc60c7/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:17:56.340163 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a4fd69-f7ab-41ca-93fa-c0fa8d17aabe None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:56.340479 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a4fd69-f7ab-41ca-93fa-c0fa8d17aabe None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:56.340795 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a4fd69-f7ab-41ca-93fa-c0fa8d17aabe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:56.363498 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-12a4fd69-f7ab-41ca-93fa-c0fa8d17aabe None None] MFA Rules not processed for user `2921386cc370487da26a786c8198c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:56.364794 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-12a4fd69-f7ab-41ca-93fa-c0fa8d17aabe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:56.407693 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-ee6c0cd1-cdd1-4a42-a4bb-7890fcda053d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c66250df2e054415b517b2de8857bc87', 'request_id': 'req-ee6c0cd1-cdd1-4a42-a4bb-7890fcda053d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:56.424179 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-ee6c0cd1-cdd1-4a42-a4bb-7890fcda053d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c66250df2e054415b517b2de8857bc87', 'request_id': 'req-ee6c0cd1-cdd1-4a42-a4bb-7890fcda053d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:56.427077 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:56.434907 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1041/2073] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:17:56 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:56.436046 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a4fd69-f7ab-41ca-93fa-c0fa8d17aabe None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:56.464945 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-ee6c0cd1-cdd1-4a42-a4bb-7890fcda053d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c66250df2e054415b517b2de8857bc87', 'request_id': 'req-ee6c0cd1-cdd1-4a42-a4bb-7890fcda053d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:56.491387 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-ee6c0cd1-cdd1-4a42-a4bb-7890fcda053d admin admin] The token cache is being invalidate because project c66250df2e054415b517b2de8857bc87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:17:56.491682 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-ee6c0cd1-cdd1-4a42-a4bb-7890fcda053d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:56.494262 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-ee6c0cd1-cdd1-4a42-a4bb-7890fcda053d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:56.504331 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1033/2074] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:17:56 2022] DELETE /identity/v3/projects/c66250df2e054415b517b2de8857bc87 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:56.510105 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbba7e2-4cc2-494d-abce-b448c633db6b None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:56.510390 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbba7e2-4cc2-494d-abce-b448c633db6b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:56.510666 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbba7e2-4cc2-494d-abce-b448c633db6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:56.545986 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-dbbba7e2-4cc2-494d-abce-b448c633db6b None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:56.547402 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-dbbba7e2-4cc2-494d-abce-b448c633db6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:56.573630 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a4fd69-f7ab-41ca-93fa-c0fa8d17aabe None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:56.658475 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8099fc-4a20-438a-a75d-5d471a462dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:56.659323 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8099fc-4a20-438a-a75d-5d471a462dbf admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:56.659650 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8099fc-4a20-438a-a75d-5d471a462dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:56.659919 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8099fc-4a20-438a-a75d-5d471a462dbf admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:56.660790 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8099fc-4a20-438a-a75d-5d471a462dbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:56.663225 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8099fc-4a20-438a-a75d-5d471a462dbf admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:56.683763 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1042/2075] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:17:56 2022] GET /identity/v3/roles => generated 1420 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:17:56.685737 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8099fc-4a20-438a-a75d-5d471a462dbf admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:56.689106 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-4d8099fc-4a20-438a-a75d-5d471a462dbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:56.717066 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1034/2076] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:17:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:56.717329 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8099fc-4a20-438a-a75d-5d471a462dbf admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:56.718753 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eda780-14aa-4feb-a226-75a93f323fcc None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:56.719120 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eda780-14aa-4feb-a226-75a93f323fcc None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:56.719404 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eda780-14aa-4feb-a226-75a93f323fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:56.730569 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182f4da3-b6be-414a-ae59-506b435e0bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:56.733224 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f4da3-b6be-414a-ae59-506b435e0bc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:56.733854 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f4da3-b6be-414a-ae59-506b435e0bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:56.734152 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f4da3-b6be-414a-ae59-506b435e0bc6 admin admin] PATH_INFO: `/v3/projects/009f1033f43945cd8483afafd38f32b3/users/a280595c6ff4495ab9f4120b4bcc60c7/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:56.739860 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182f4da3-b6be-414a-ae59-506b435e0bc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=009f1033f43945cd8483afafd38f32b3, user_id=a280595c6ff4495ab9f4120b4bcc60c7, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:56.747519 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182f4da3-b6be-414a-ae59-506b435e0bc6 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:56.766254 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-c2eda780-14aa-4feb-a226-75a93f323fcc None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:56.767715 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c2eda780-14aa-4feb-a226-75a93f323fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:56.777292 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1043/2077] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:17:56 2022] PUT /identity/v3/projects/009f1033f43945cd8483afafd38f32b3/users/a280595c6ff4495ab9f4120b4bcc60c7/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:56.889411 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1035/2078] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:17:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:17:56.893377 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182f4da3-b6be-414a-ae59-506b435e0bc6 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:56.908087 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182f4da3-b6be-414a-ae59-506b435e0bc6 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:56.914335 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cfacf3-933a-4c6f-9e66-df570519aa10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:56.914904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cfacf3-933a-4c6f-9e66-df570519aa10 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:56.915236 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cfacf3-933a-4c6f-9e66-df570519aa10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:56.915513 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cfacf3-933a-4c6f-9e66-df570519aa10 admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:56.916477 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cfacf3-933a-4c6f-9e66-df570519aa10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:17:56.916966 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cfacf3-933a-4c6f-9e66-df570519aa10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:56.919254 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cfacf3-933a-4c6f-9e66-df570519aa10 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:56.951441 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1044/2079] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 6 16:17:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:17:56.954145 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eda780-14aa-4feb-a226-75a93f323fcc None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:56.966671 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eda780-14aa-4feb-a226-75a93f323fcc None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:56.973784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be7617f-46ed-46f9-a12b-027045cd6661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:56.974477 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be7617f-46ed-46f9-a12b-027045cd6661 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:56.974985 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be7617f-46ed-46f9-a12b-027045cd6661 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:56.975412 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be7617f-46ed-46f9-a12b-027045cd6661 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:56.976184 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be7617f-46ed-46f9-a12b-027045cd6661 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:56.978308 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be7617f-46ed-46f9-a12b-027045cd6661 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:57.018009 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1036/2080] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:17:56 2022] POST /identity/v3/projects => generated 371 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:17:57.024417 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cfacf3-933a-4c6f-9e66-df570519aa10 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:57.041232 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:57.047445 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cfacf3-933a-4c6f-9e66-df570519aa10 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:57.123480 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0745cb2-945c-4b10-8a12-f2b62cb16213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:57.124297 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0745cb2-945c-4b10-8a12-f2b62cb16213 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:57.124632 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0745cb2-945c-4b10-8a12-f2b62cb16213 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:57.124991 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0745cb2-945c-4b10-8a12-f2b62cb16213 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:57.125787 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0745cb2-945c-4b10-8a12-f2b62cb16213 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:57.127783 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0745cb2-945c-4b10-8a12-f2b62cb16213 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:57.155025 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1045/2081] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:17:57 2022] POST /identity/v3/users => generated 286 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:17:57.159897 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be7617f-46ed-46f9-a12b-027045cd6661 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:57.173452 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be7617f-46ed-46f9-a12b-027045cd6661 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:57.180788 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8330b4-d31b-4b4a-a7dc-70d121ae5a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:57.181879 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8330b4-d31b-4b4a-a7dc-70d121ae5a3a admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:57.182371 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8330b4-d31b-4b4a-a7dc-70d121ae5a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:57.182721 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8330b4-d31b-4b4a-a7dc-70d121ae5a3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:57.183561 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8330b4-d31b-4b4a-a7dc-70d121ae5a3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:57.185355 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8330b4-d31b-4b4a-a7dc-70d121ae5a3a admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:57.201408 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1037/2082] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:17:57 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:17:57.205828 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0745cb2-945c-4b10-8a12-f2b62cb16213 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:57.219237 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0745cb2-945c-4b10-8a12-f2b62cb16213 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:57.225490 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb1c19c-71bd-44a2-924e-e89478542209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:57.226311 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb1c19c-71bd-44a2-924e-e89478542209 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:57.226895 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb1c19c-71bd-44a2-924e-e89478542209 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:57.227043 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb1c19c-71bd-44a2-924e-e89478542209 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:57.228256 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb1c19c-71bd-44a2-924e-e89478542209 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:57.229397 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb1c19c-71bd-44a2-924e-e89478542209 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:57.241652 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1046/2083] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:17:57 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:17:57.256430 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8330b4-d31b-4b4a-a7dc-70d121ae5a3a admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:57.283037 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8330b4-d31b-4b4a-a7dc-70d121ae5a3a admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:57.290588 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cae089-36a2-4a1e-900f-7011c632c819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:57.291666 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cae089-36a2-4a1e-900f-7011c632c819 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:57.291889 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cae089-36a2-4a1e-900f-7011c632c819 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:57.292252 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cae089-36a2-4a1e-900f-7011c632c819 admin admin] PATH_INFO: `/v3/projects/0fe959bc8c9a409a94150e6683faff45/users/7865b032e8a5471c9727a82aa5278647/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:57.315624 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cae089-36a2-4a1e-900f-7011c632c819 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fe959bc8c9a409a94150e6683faff45, user_id=7865b032e8a5471c9727a82aa5278647, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:57.318591 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cae089-36a2-4a1e-900f-7011c632c819 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:57.349580 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1038/2084] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:17:57 2022] PUT /identity/v3/projects/0fe959bc8c9a409a94150e6683faff45/users/7865b032e8a5471c9727a82aa5278647/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:57.351396 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74285de4-02ce-4fd9-9152-2a6ff22b7234 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:57.351696 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74285de4-02ce-4fd9-9152-2a6ff22b7234 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:57.352008 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74285de4-02ce-4fd9-9152-2a6ff22b7234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:57.367347 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cae089-36a2-4a1e-900f-7011c632c819 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:57.369759 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a3cae089-36a2-4a1e-900f-7011c632c819 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:57.417412 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-74285de4-02ce-4fd9-9152-2a6ff22b7234 None None] MFA Rules not processed for user `7865b032e8a5471c9727a82aa5278647`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:57.420659 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-74285de4-02ce-4fd9-9152-2a6ff22b7234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:57.452032 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:DELETE=1,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:57.504721 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cae089-36a2-4a1e-900f-7011c632c819 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:57.540947 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beabf66-3fa9-4620-806e-7a66ddd55f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:57.543462 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beabf66-3fa9-4620-806e-7a66ddd55f9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:57.543748 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beabf66-3fa9-4620-806e-7a66ddd55f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:57.544730 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beabf66-3fa9-4620-806e-7a66ddd55f9d admin admin] PATH_INFO: `/v3/users/d3baa1fea74144b5b170613eedfe44ae` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:57.549618 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beabf66-3fa9-4620-806e-7a66ddd55f9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3baa1fea74144b5b170613eedfe44ae)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:57.554019 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beabf66-3fa9-4620-806e-7a66ddd55f9d admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:57.600889 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1047/2085] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:17:57 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:17:57.613680 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74285de4-02ce-4fd9-9152-2a6ff22b7234 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:57.615988 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-74285de4-02ce-4fd9-9152-2a6ff22b7234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:57.650718 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1beabf66-3fa9-4620-806e-7a66ddd55f9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3baa1fea74144b5b170613eedfe44ae'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:57.651725 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1beabf66-3fa9-4620-806e-7a66ddd55f9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3baa1fea74144b5b170613eedfe44ae'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:57.661214 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1beabf66-3fa9-4620-806e-7a66ddd55f9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3baa1fea74144b5b170613eedfe44ae'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:57.677496 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1beabf66-3fa9-4620-806e-7a66ddd55f9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3baa1fea74144b5b170613eedfe44ae'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:57.688619 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1beabf66-3fa9-4620-806e-7a66ddd55f9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3baa1fea74144b5b170613eedfe44ae'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:57.724832 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1039/2086] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:17:57 2022] DELETE /identity/v3/users/d3baa1fea74144b5b170613eedfe44ae => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:17:57.729076 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712050f8-6450-443c-b9e3-6be5490d91a7 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:57.730419 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712050f8-6450-443c-b9e3-6be5490d91a7 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:57.730894 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712050f8-6450-443c-b9e3-6be5490d91a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:57.763195 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74285de4-02ce-4fd9-9152-2a6ff22b7234 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:57.767497 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-712050f8-6450-443c-b9e3-6be5490d91a7 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:57.771410 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-712050f8-6450-443c-b9e3-6be5490d91a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:57.776331 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:57.854274 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4ee37f-01c6-4e4d-9d39-7f05a0ef7ac5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63de41f4348243998029f7d07aa5dc40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:57.855131 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ee37f-01c6-4e4d-9d39-7f05a0ef7ac5 service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:57.855611 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ee37f-01c6-4e4d-9d39-7f05a0ef7ac5 service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:57.856088 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ee37f-01c6-4e4d-9d39-7f05a0ef7ac5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:57.858233 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3e4ee37f-01c6-4e4d-9d39-7f05a0ef7ac5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:57.858643 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1040/2087] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:17:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:17:57.868810 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712050f8-6450-443c-b9e3-6be5490d91a7 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:57.870205 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-712050f8-6450-443c-b9e3-6be5490d91a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:57.944973 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4ee37f-01c6-4e4d-9d39-7f05a0ef7ac5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:57.946742 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4ee37f-01c6-4e4d-9d39-7f05a0ef7ac5 service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:57.957790 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1048/2088] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:17:57 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:17:58.040899 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712050f8-6450-443c-b9e3-6be5490d91a7 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:58.054524 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c55b6f-c471-40db-9280-31b6df485ffc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:58.058562 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c55b6f-c471-40db-9280-31b6df485ffc service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:58.059995 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c55b6f-c471-40db-9280-31b6df485ffc service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:58.060632 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c55b6f-c471-40db-9280-31b6df485ffc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:58.078210 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:58.088312 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c55b6f-c471-40db-9280-31b6df485ffc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:58.097707 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c55b6f-c471-40db-9280-31b6df485ffc service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:58.149304 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1041/2089] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:17:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:17:58.433799 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4ee37f-01c6-4e4d-9d39-7f05a0ef7ac5 service nova] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:58.433799 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3e4ee37f-01c6-4e4d-9d39-7f05a0ef7ac5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:58.473394 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4ee37f-01c6-4e4d-9d39-7f05a0ef7ac5 service nova] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:58.481018 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b5dd35-1623-4192-a789-9ced8f059bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:58.481851 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b5dd35-1623-4192-a789-9ced8f059bcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:58.482365 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b5dd35-1623-4192-a789-9ced8f059bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:58.482736 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b5dd35-1623-4192-a789-9ced8f059bcf admin admin] PATH_INFO: `/v3/users/7865b032e8a5471c9727a82aa5278647` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:58.484143 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b5dd35-1623-4192-a789-9ced8f059bcf admin admin] RBAC: Authorizing `identity:delete_user(user_id=7865b032e8a5471c9727a82aa5278647)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:58.486656 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b5dd35-1623-4192-a789-9ced8f059bcf admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:58.660505 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-c3b5dd35-1623-4192-a789-9ced8f059bcf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7865b032e8a5471c9727a82aa5278647'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:58.661789 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-c3b5dd35-1623-4192-a789-9ced8f059bcf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7865b032e8a5471c9727a82aa5278647'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:58.679637 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-c3b5dd35-1623-4192-a789-9ced8f059bcf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7865b032e8a5471c9727a82aa5278647'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:58.718214 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-c3b5dd35-1623-4192-a789-9ced8f059bcf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7865b032e8a5471c9727a82aa5278647'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:58.720429 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-c3b5dd35-1623-4192-a789-9ced8f059bcf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7865b032e8a5471c9727a82aa5278647'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:58.740892 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1049/2090] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:17:58 2022] DELETE /identity/v3/users/7865b032e8a5471c9727a82aa5278647 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:58.745370 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6311ff-eb71-4990-9cea-351c756a9050 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:58.745647 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6311ff-eb71-4990-9cea-351c756a9050 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:58.746108 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6311ff-eb71-4990-9cea-351c756a9050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:58.773800 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-bd6311ff-eb71-4990-9cea-351c756a9050 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:58.775059 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-bd6311ff-eb71-4990-9cea-351c756a9050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:58.889516 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1042/2091] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:17:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:17:58.915346 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b5dd35-1623-4192-a789-9ced8f059bcf admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:58.918356 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-c3b5dd35-1623-4192-a789-9ced8f059bcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:59.012451 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:59.077164 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b5dd35-1623-4192-a789-9ced8f059bcf admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:59.090627 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f265959c-0183-4e58-859f-596e9013cd60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:59.091272 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f265959c-0183-4e58-859f-596e9013cd60 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:59.091547 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f265959c-0183-4e58-859f-596e9013cd60 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:59.091855 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f265959c-0183-4e58-859f-596e9013cd60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:59.110798 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f265959c-0183-4e58-859f-596e9013cd60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:59.112937 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f265959c-0183-4e58-859f-596e9013cd60 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:59.136885 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1050/2092] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:17:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:17:59.325663 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6311ff-eb71-4990-9cea-351c756a9050 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:59.327387 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-bd6311ff-eb71-4990-9cea-351c756a9050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:59.349014 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6311ff-eb71-4990-9cea-351c756a9050 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:59.361504 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878a8e23-b62e-45ff-9b5d-2bca34c74a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:59.362175 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878a8e23-b62e-45ff-9b5d-2bca34c74a1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:59.362483 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878a8e23-b62e-45ff-9b5d-2bca34c74a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:59.362740 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878a8e23-b62e-45ff-9b5d-2bca34c74a1c admin admin] PATH_INFO: `/v3/projects/f6e89f64f5404bda96cf64d754cbcbb2` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:59.366765 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878a8e23-b62e-45ff-9b5d-2bca34c74a1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6e89f64f5404bda96cf64d754cbcbb2)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:59.371010 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878a8e23-b62e-45ff-9b5d-2bca34c74a1c admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:59.485844 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-878a8e23-b62e-45ff-9b5d-2bca34c74a1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6e89f64f5404bda96cf64d754cbcbb2', 'request_id': 'req-878a8e23-b62e-45ff-9b5d-2bca34c74a1c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:59.504509 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-878a8e23-b62e-45ff-9b5d-2bca34c74a1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6e89f64f5404bda96cf64d754cbcbb2', 'request_id': 'req-878a8e23-b62e-45ff-9b5d-2bca34c74a1c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:59.513926 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-878a8e23-b62e-45ff-9b5d-2bca34c74a1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6e89f64f5404bda96cf64d754cbcbb2', 'request_id': 'req-878a8e23-b62e-45ff-9b5d-2bca34c74a1c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:59.538397 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-878a8e23-b62e-45ff-9b5d-2bca34c74a1c admin admin] The token cache is being invalidate because project f6e89f64f5404bda96cf64d754cbcbb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:17:59.538690 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-878a8e23-b62e-45ff-9b5d-2bca34c74a1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:59.540187 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-878a8e23-b62e-45ff-9b5d-2bca34c74a1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:59.547235 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f265959c-0183-4e58-859f-596e9013cd60 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:59.547761 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1043/2093] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:17:59 2022] DELETE /identity/v3/projects/f6e89f64f5404bda96cf64d754cbcbb2 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:59.559521 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f265959c-0183-4e58-859f-596e9013cd60 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:59.705040 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13fcc4b-1fcd-424d-9169-b32dddf49731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:59.707871 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13fcc4b-1fcd-424d-9169-b32dddf49731 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:59.708428 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13fcc4b-1fcd-424d-9169-b32dddf49731 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:59.709129 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13fcc4b-1fcd-424d-9169-b32dddf49731 admin admin] PATH_INFO: `/v3/users/390ca7ac2b9e4855aad3d1ed9ebd9e5e` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:59.710910 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13fcc4b-1fcd-424d-9169-b32dddf49731 admin admin] RBAC: Authorizing `identity:delete_user(user_id=390ca7ac2b9e4855aad3d1ed9ebd9e5e)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:59.713693 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13fcc4b-1fcd-424d-9169-b32dddf49731 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:59.842979 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a13fcc4b-1fcd-424d-9169-b32dddf49731 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '390ca7ac2b9e4855aad3d1ed9ebd9e5e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:59.844235 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a13fcc4b-1fcd-424d-9169-b32dddf49731 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '390ca7ac2b9e4855aad3d1ed9ebd9e5e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:59.864424 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a13fcc4b-1fcd-424d-9169-b32dddf49731 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '390ca7ac2b9e4855aad3d1ed9ebd9e5e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:59.895656 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a13fcc4b-1fcd-424d-9169-b32dddf49731 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '390ca7ac2b9e4855aad3d1ed9ebd9e5e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:59.896952 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a13fcc4b-1fcd-424d-9169-b32dddf49731 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '390ca7ac2b9e4855aad3d1ed9ebd9e5e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:59.925856 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1051/2094] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:17:59 2022] DELETE /identity/v3/users/390ca7ac2b9e4855aad3d1ed9ebd9e5e => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:18:00.348657 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878a8e23-b62e-45ff-9b5d-2bca34c74a1c admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:00.357770 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-878a8e23-b62e-45ff-9b5d-2bca34c74a1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:00.372468 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878a8e23-b62e-45ff-9b5d-2bca34c74a1c admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:00.465614 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41f81a2-5ae8-41d6-b280-f51f757eb943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:00.466537 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41f81a2-5ae8-41d6-b280-f51f757eb943 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:00.466954 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41f81a2-5ae8-41d6-b280-f51f757eb943 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:00.467366 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41f81a2-5ae8-41d6-b280-f51f757eb943 admin admin] PATH_INFO: `/v3/projects/0fe959bc8c9a409a94150e6683faff45` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:00.469229 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41f81a2-5ae8-41d6-b280-f51f757eb943 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fe959bc8c9a409a94150e6683faff45)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:00.471541 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41f81a2-5ae8-41d6-b280-f51f757eb943 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:00.505104 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:18:00.573803 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b41f81a2-5ae8-41d6-b280-f51f757eb943 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fe959bc8c9a409a94150e6683faff45', 'request_id': 'req-b41f81a2-5ae8-41d6-b280-f51f757eb943'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:00.592845 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b41f81a2-5ae8-41d6-b280-f51f757eb943 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fe959bc8c9a409a94150e6683faff45', 'request_id': 'req-b41f81a2-5ae8-41d6-b280-f51f757eb943'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:00.605551 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b41f81a2-5ae8-41d6-b280-f51f757eb943 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fe959bc8c9a409a94150e6683faff45', 'request_id': 'req-b41f81a2-5ae8-41d6-b280-f51f757eb943'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:00.626838 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b41f81a2-5ae8-41d6-b280-f51f757eb943 admin admin] The token cache is being invalidate because project 0fe959bc8c9a409a94150e6683faff45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:18:00.627239 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b41f81a2-5ae8-41d6-b280-f51f757eb943 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:00.627991 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b41f81a2-5ae8-41d6-b280-f51f757eb943 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:00.633850 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1044/2095] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:18:00 2022] DELETE /identity/v3/projects/0fe959bc8c9a409a94150e6683faff45 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:18:00.636024 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e912a4b8-de9a-4e42-9c95-6a0698b691af None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:00.636485 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e912a4b8-de9a-4e42-9c95-6a0698b691af None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:00.636935 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e912a4b8-de9a-4e42-9c95-6a0698b691af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:00.685431 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-e912a4b8-de9a-4e42-9c95-6a0698b691af None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:18:00.687768 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e912a4b8-de9a-4e42-9c95-6a0698b691af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:00.728901 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41f81a2-5ae8-41d6-b280-f51f757eb943 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:00.730863 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b41f81a2-5ae8-41d6-b280-f51f757eb943 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:00.750952 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:18:00.788871 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1052/2096] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:18:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:18:00.794707 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4427c16-d2cc-4282-849a-b124b863e21b None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:00.795304 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4427c16-d2cc-4282-849a-b124b863e21b None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:00.796977 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4427c16-d2cc-4282-849a-b124b863e21b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:00.826990 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-c4427c16-d2cc-4282-849a-b124b863e21b None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:18:00.828641 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-c4427c16-d2cc-4282-849a-b124b863e21b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:00.839615 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1053/2097] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:18:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:18:00.845841 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4427c16-d2cc-4282-849a-b124b863e21b None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:00.860721 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4427c16-d2cc-4282-849a-b124b863e21b None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:00.866135 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41f81a2-5ae8-41d6-b280-f51f757eb943 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:00.868824 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc237a34-ad91-4766-b57c-9824e061e1ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:00.869791 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc237a34-ad91-4766-b57c-9824e061e1ca admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:00.870243 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc237a34-ad91-4766-b57c-9824e061e1ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:00.870658 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc237a34-ad91-4766-b57c-9824e061e1ca admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:00.871731 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc237a34-ad91-4766-b57c-9824e061e1ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:18:00.872466 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc237a34-ad91-4766-b57c-9824e061e1ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:00.875219 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc237a34-ad91-4766-b57c-9824e061e1ca admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:00.875327 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ef2dad-0e38-4e0a-99ba-10cc7dae8207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:00.876487 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ef2dad-0e38-4e0a-99ba-10cc7dae8207 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:00.876785 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ef2dad-0e38-4e0a-99ba-10cc7dae8207 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:00.877075 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ef2dad-0e38-4e0a-99ba-10cc7dae8207 admin admin] PATH_INFO: `/v3/projects/7320935cc63049dea292fefa5e6a5c91` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:00.881574 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ef2dad-0e38-4e0a-99ba-10cc7dae8207 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7320935cc63049dea292fefa5e6a5c91)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:00.884224 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ef2dad-0e38-4e0a-99ba-10cc7dae8207 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:00.897677 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1054/2098] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 6 16:18:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:18:00.899410 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc237a34-ad91-4766-b57c-9824e061e1ca admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:00.912151 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc237a34-ad91-4766-b57c-9824e061e1ca admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:00.918264 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc3aa4e-137f-4db7-b5df-6f3d48f5e59e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:00.919505 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc3aa4e-137f-4db7-b5df-6f3d48f5e59e admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:00.919884 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc3aa4e-137f-4db7-b5df-6f3d48f5e59e admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:00.920628 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc3aa4e-137f-4db7-b5df-6f3d48f5e59e admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:00.921434 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc3aa4e-137f-4db7-b5df-6f3d48f5e59e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:00.924069 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc3aa4e-137f-4db7-b5df-6f3d48f5e59e admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:00.965938 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-07ef2dad-0e38-4e0a-99ba-10cc7dae8207 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7320935cc63049dea292fefa5e6a5c91', 'request_id': 'req-07ef2dad-0e38-4e0a-99ba-10cc7dae8207'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:00.970551 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1055/2099] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:18:00 2022] POST /identity/v3/projects => generated 357 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:18:00.972548 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc3aa4e-137f-4db7-b5df-6f3d48f5e59e admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:00.976323 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-07ef2dad-0e38-4e0a-99ba-10cc7dae8207 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7320935cc63049dea292fefa5e6a5c91', 'request_id': 'req-07ef2dad-0e38-4e0a-99ba-10cc7dae8207'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:00.986900 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-07ef2dad-0e38-4e0a-99ba-10cc7dae8207 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7320935cc63049dea292fefa5e6a5c91', 'request_id': 'req-07ef2dad-0e38-4e0a-99ba-10cc7dae8207'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:01.003161 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-07ef2dad-0e38-4e0a-99ba-10cc7dae8207 admin admin] The token cache is being invalidate because project 7320935cc63049dea292fefa5e6a5c91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:18:01.003455 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-07ef2dad-0e38-4e0a-99ba-10cc7dae8207 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:01.004079 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-07ef2dad-0e38-4e0a-99ba-10cc7dae8207 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:01.006781 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1045/2100] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:18:00 2022] DELETE /identity/v3/projects/7320935cc63049dea292fefa5e6a5c91 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:18:01.010857 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ef2dad-0e38-4e0a-99ba-10cc7dae8207 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:01.014141 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-07ef2dad-0e38-4e0a-99ba-10cc7dae8207 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:01.066513 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc3aa4e-137f-4db7-b5df-6f3d48f5e59e admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:01.076148 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83ad7a9-96cc-4cb7-a97a-63f6920bd7be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:01.077511 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ad7a9-96cc-4cb7-a97a-63f6920bd7be admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:01.078160 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ad7a9-96cc-4cb7-a97a-63f6920bd7be admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:01.079288 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ad7a9-96cc-4cb7-a97a-63f6920bd7be admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:01.080622 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83ad7a9-96cc-4cb7-a97a-63f6920bd7be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:01.085041 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:18:01.086085 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83ad7a9-96cc-4cb7-a97a-63f6920bd7be admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:01.122348 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ef2dad-0e38-4e0a-99ba-10cc7dae8207 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:01.128074 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1056/2101] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:18:00 2022] POST /identity/v3/users => generated 279 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:18:01.131320 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47356bab-3088-4aeb-b1fb-34b0cc791366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:01.132293 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47356bab-3088-4aeb-b1fb-34b0cc791366 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:01.132980 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47356bab-3088-4aeb-b1fb-34b0cc791366 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:01.132980 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47356bab-3088-4aeb-b1fb-34b0cc791366 admin admin] PATH_INFO: `/v3/users/4fb91f49e05e4e4bab85a10f8a4e9f48` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:01.133415 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83ad7a9-96cc-4cb7-a97a-63f6920bd7be admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:01.134563 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47356bab-3088-4aeb-b1fb-34b0cc791366 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fb91f49e05e4e4bab85a10f8a4e9f48)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:01.137059 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e83ad7a9-96cc-4cb7-a97a-63f6920bd7be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:01.137241 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47356bab-3088-4aeb-b1fb-34b0cc791366 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:01.154618 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83ad7a9-96cc-4cb7-a97a-63f6920bd7be admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:01.161934 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e4e285-9f95-4ecb-960e-6aeff7cca27b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:01.163008 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e4e285-9f95-4ecb-960e-6aeff7cca27b admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:01.163444 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e4e285-9f95-4ecb-960e-6aeff7cca27b admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:01.163900 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e4e285-9f95-4ecb-960e-6aeff7cca27b admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:01.164714 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e4e285-9f95-4ecb-960e-6aeff7cca27b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:01.166958 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e4e285-9f95-4ecb-960e-6aeff7cca27b admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:01.181644 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1057/2102] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:18:01 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:18:01.190214 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e4e285-9f95-4ecb-960e-6aeff7cca27b admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:01.208386 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e4e285-9f95-4ecb-960e-6aeff7cca27b admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:01.214233 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0ef543-de53-4d9a-a59b-c460177d109e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:01.214856 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0ef543-de53-4d9a-a59b-c460177d109e admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:01.215312 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0ef543-de53-4d9a-a59b-c460177d109e admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:01.215420 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0ef543-de53-4d9a-a59b-c460177d109e admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:01.216055 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0ef543-de53-4d9a-a59b-c460177d109e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:01.217738 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0ef543-de53-4d9a-a59b-c460177d109e admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:01.225991 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-47356bab-3088-4aeb-b1fb-34b0cc791366 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fb91f49e05e4e4bab85a10f8a4e9f48'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:01.229527 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-47356bab-3088-4aeb-b1fb-34b0cc791366 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fb91f49e05e4e4bab85a10f8a4e9f48'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:01.235310 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1058/2103] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:18:01 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:18:01.244839 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0ef543-de53-4d9a-a59b-c460177d109e admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:01.255163 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-47356bab-3088-4aeb-b1fb-34b0cc791366 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fb91f49e05e4e4bab85a10f8a4e9f48'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:01.263946 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0ef543-de53-4d9a-a59b-c460177d109e admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:01.284942 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7524a92a-e9e7-4747-9566-95266b2355e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:01.284942 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7524a92a-e9e7-4747-9566-95266b2355e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:01.284942 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7524a92a-e9e7-4747-9566-95266b2355e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:01.284942 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7524a92a-e9e7-4747-9566-95266b2355e1 admin admin] PATH_INFO: `/v3/projects/12154ec79bc84b0b899de1bc94db3982/users/a320a7e5bd7e4fd792d196a2745c8a1f/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:01.295909 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-47356bab-3088-4aeb-b1fb-34b0cc791366 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fb91f49e05e4e4bab85a10f8a4e9f48'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:01.308714 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7524a92a-e9e7-4747-9566-95266b2355e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12154ec79bc84b0b899de1bc94db3982, user_id=a320a7e5bd7e4fd792d196a2745c8a1f, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:01.312158 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-47356bab-3088-4aeb-b1fb-34b0cc791366 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fb91f49e05e4e4bab85a10f8a4e9f48'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:01.314159 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7524a92a-e9e7-4747-9566-95266b2355e1 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:01.329239 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1046/2104] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:18:01 2022] DELETE /identity/v3/users/4fb91f49e05e4e4bab85a10f8a4e9f48 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:18:01.341356 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1059/2105] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:18:01 2022] PUT /identity/v3/projects/12154ec79bc84b0b899de1bc94db3982/users/a320a7e5bd7e4fd792d196a2745c8a1f/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:18:01.352161 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3df126-a803-4ea4-a7bc-f9e2a698cb08 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:01.352662 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3df126-a803-4ea4-a7bc-f9e2a698cb08 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:01.352795 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3df126-a803-4ea4-a7bc-f9e2a698cb08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:01.391389 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-3c3df126-a803-4ea4-a7bc-f9e2a698cb08 None None] MFA Rules not processed for user `a320a7e5bd7e4fd792d196a2745c8a1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:18:01.392653 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-3c3df126-a803-4ea4-a7bc-f9e2a698cb08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:01.508101 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1047/2106] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:18:01 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:18:01.515278 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7524a92a-e9e7-4747-9566-95266b2355e1 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:01.518373 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-7524a92a-e9e7-4747-9566-95266b2355e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:01.651958 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:18:01.692312 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7524a92a-e9e7-4747-9566-95266b2355e1 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:01.703706 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa63e65-77c1-4350-932c-6ae9f17d59da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:01.705277 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa63e65-77c1-4350-932c-6ae9f17d59da admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:01.721543 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa63e65-77c1-4350-932c-6ae9f17d59da admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:01.722067 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa63e65-77c1-4350-932c-6ae9f17d59da admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:01.725683 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa63e65-77c1-4350-932c-6ae9f17d59da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:01.729162 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa63e65-77c1-4350-932c-6ae9f17d59da admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:01.778466 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3df126-a803-4ea4-a7bc-f9e2a698cb08 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:01.778683 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1060/2107] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:18:01 2022] POST /identity/v3/projects => generated 355 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:18:01.799046 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3df126-a803-4ea4-a7bc-f9e2a698cb08 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:01.887961 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:18:01.911017 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4016ee-ef40-467c-bcd3-78c9ea5421cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:01.911925 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4016ee-ef40-467c-bcd3-78c9ea5421cf admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:01.912141 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4016ee-ef40-467c-bcd3-78c9ea5421cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:01.912412 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4016ee-ef40-467c-bcd3-78c9ea5421cf admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:01.913098 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4016ee-ef40-467c-bcd3-78c9ea5421cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:01.915077 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4016ee-ef40-467c-bcd3-78c9ea5421cf admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:01.944649 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1048/2108] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:18:01 2022] POST /identity/v3/users => generated 284 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:18:01.945970 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa63e65-77c1-4350-932c-6ae9f17d59da admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:01.959980 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa63e65-77c1-4350-932c-6ae9f17d59da admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:01.965761 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc7d4ef-7002-4152-acbd-1f05b296d1f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:01.966446 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc7d4ef-7002-4152-acbd-1f05b296d1f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:01.966689 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc7d4ef-7002-4152-acbd-1f05b296d1f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:01.967078 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc7d4ef-7002-4152-acbd-1f05b296d1f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:01.967803 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc7d4ef-7002-4152-acbd-1f05b296d1f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:01.970793 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc7d4ef-7002-4152-acbd-1f05b296d1f7 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:01.984801 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1061/2109] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:18:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 16:18:01.986815 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4016ee-ef40-467c-bcd3-78c9ea5421cf admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:02.000464 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4016ee-ef40-467c-bcd3-78c9ea5421cf admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:02.007648 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037bf8a3-ba3f-42f8-b95f-714d16a572d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:02.009311 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037bf8a3-ba3f-42f8-b95f-714d16a572d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:02.009748 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037bf8a3-ba3f-42f8-b95f-714d16a572d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:02.010519 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037bf8a3-ba3f-42f8-b95f-714d16a572d0 admin admin] PATH_INFO: `/v3/projects/0812168eb3be4dfeaf87a0ac9a2afd0c/users/bab8bcc84152488e937c3dc603842bac/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:02.031357 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037bf8a3-ba3f-42f8-b95f-714d16a572d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0812168eb3be4dfeaf87a0ac9a2afd0c, user_id=bab8bcc84152488e937c3dc603842bac, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:02.034678 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037bf8a3-ba3f-42f8-b95f-714d16a572d0 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:02.055467 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1049/2110] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:18:01 2022] PUT /identity/v3/projects/0812168eb3be4dfeaf87a0ac9a2afd0c/users/bab8bcc84152488e937c3dc603842bac/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:18:02.059617 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc7d4ef-7002-4152-acbd-1f05b296d1f7 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:02.072858 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc7d4ef-7002-4152-acbd-1f05b296d1f7 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:02.171760 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5920c0-6cab-42bd-8694-c4edfd95b73e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:02.172530 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5920c0-6cab-42bd-8694-c4edfd95b73e admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:02.172914 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5920c0-6cab-42bd-8694-c4edfd95b73e admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:02.173260 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5920c0-6cab-42bd-8694-c4edfd95b73e admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:02.173985 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5920c0-6cab-42bd-8694-c4edfd95b73e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:02.175776 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5920c0-6cab-42bd-8694-c4edfd95b73e admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:02.189686 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1062/2111] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:18:02 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:18:02.194349 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037bf8a3-ba3f-42f8-b95f-714d16a572d0 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:02.217481 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037bf8a3-ba3f-42f8-b95f-714d16a572d0 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:02.224576 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef6edfd-ede2-4db5-a84e-de12904febd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:02.225431 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef6edfd-ede2-4db5-a84e-de12904febd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:02.225793 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef6edfd-ede2-4db5-a84e-de12904febd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:02.226070 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef6edfd-ede2-4db5-a84e-de12904febd0 admin admin] PATH_INFO: `/v3/projects/0812168eb3be4dfeaf87a0ac9a2afd0c/users/bab8bcc84152488e937c3dc603842bac/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:02.228521 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef6edfd-ede2-4db5-a84e-de12904febd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0812168eb3be4dfeaf87a0ac9a2afd0c, user_id=bab8bcc84152488e937c3dc603842bac, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:02.231656 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef6edfd-ede2-4db5-a84e-de12904febd0 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:02.243744 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5920c0-6cab-42bd-8694-c4edfd95b73e admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:02.247550 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1050/2112] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:18:02 2022] PUT /identity/v3/projects/0812168eb3be4dfeaf87a0ac9a2afd0c/users/bab8bcc84152488e937c3dc603842bac/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:18:02.251834 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-af5920c0-6cab-42bd-8694-c4edfd95b73e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:02.259616 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10924bb1-c548-4665-a09c-2ee08f19f908 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:02.260169 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10924bb1-c548-4665-a09c-2ee08f19f908 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:02.260613 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10924bb1-c548-4665-a09c-2ee08f19f908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:02.275261 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5920c0-6cab-42bd-8694-c4edfd95b73e admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:02.313407 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-10924bb1-c548-4665-a09c-2ee08f19f908 None None] MFA Rules not processed for user `bab8bcc84152488e937c3dc603842bac`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:18:02.314694 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-10924bb1-c548-4665-a09c-2ee08f19f908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:02.383477 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28310547-f04b-4816-82fd-82462bbe98e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:02.383477 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28310547-f04b-4816-82fd-82462bbe98e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:02.383477 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28310547-f04b-4816-82fd-82462bbe98e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:02.384481 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28310547-f04b-4816-82fd-82462bbe98e9 admin admin] PATH_INFO: `/v3/projects/4cb1adafc10f46dd816f346effdbad68` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:02.385800 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28310547-f04b-4816-82fd-82462bbe98e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cb1adafc10f46dd816f346effdbad68)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:02.387879 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28310547-f04b-4816-82fd-82462bbe98e9 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:02.424954 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1051/2113] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:18:02 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:18:02.437887 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10924bb1-c548-4665-a09c-2ee08f19f908 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:02.439537 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-10924bb1-c548-4665-a09c-2ee08f19f908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:02.484533 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-28310547-f04b-4816-82fd-82462bbe98e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cb1adafc10f46dd816f346effdbad68', 'request_id': 'req-28310547-f04b-4816-82fd-82462bbe98e9'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:02.495082 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:18:02.498417 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-28310547-f04b-4816-82fd-82462bbe98e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cb1adafc10f46dd816f346effdbad68', 'request_id': 'req-28310547-f04b-4816-82fd-82462bbe98e9'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:02.516312 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-28310547-f04b-4816-82fd-82462bbe98e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cb1adafc10f46dd816f346effdbad68', 'request_id': 'req-28310547-f04b-4816-82fd-82462bbe98e9'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:02.532704 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-28310547-f04b-4816-82fd-82462bbe98e9 admin admin] The token cache is being invalidate because project 4cb1adafc10f46dd816f346effdbad68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:18:02.533252 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-28310547-f04b-4816-82fd-82462bbe98e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:02.534077 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-28310547-f04b-4816-82fd-82462bbe98e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:02.540652 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1063/2114] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:18:02 2022] DELETE /identity/v3/projects/4cb1adafc10f46dd816f346effdbad68 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:18:02.542926 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec18d84-90c3-441c-8963-559deabb6056 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:02.543323 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec18d84-90c3-441c-8963-559deabb6056 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:02.543669 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec18d84-90c3-441c-8963-559deabb6056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:02.544308 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10924bb1-c548-4665-a09c-2ee08f19f908 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:02.574230 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-9ec18d84-90c3-441c-8963-559deabb6056 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:18:02.575414 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-9ec18d84-90c3-441c-8963-559deabb6056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:02.643846 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:18:02.672172 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05459d5b-b9a6-4e8f-9b42-28495875bfe6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:02.672815 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05459d5b-b9a6-4e8f-9b42-28495875bfe6 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:02.673194 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05459d5b-b9a6-4e8f-9b42-28495875bfe6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:02.673518 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05459d5b-b9a6-4e8f-9b42-28495875bfe6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:02.675823 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-05459d5b-b9a6-4e8f-9b42-28495875bfe6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:02.684249 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1064/2115] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:18:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:18:02.693144 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f8daee-f15c-448a-b9d7-812ce9594e64 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:02.693483 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f8daee-f15c-448a-b9d7-812ce9594e64 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:02.693763 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f8daee-f15c-448a-b9d7-812ce9594e64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:02.736111 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-51f8daee-f15c-448a-b9d7-812ce9594e64 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:18:02.737493 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-51f8daee-f15c-448a-b9d7-812ce9594e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:02.746475 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1065/2116] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:18:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:18:02.753217 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f8daee-f15c-448a-b9d7-812ce9594e64 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:02.763563 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f8daee-f15c-448a-b9d7-812ce9594e64 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:02.777893 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eeee14-a390-4a93-bbce-a668325d2a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:02.790884 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eeee14-a390-4a93-bbce-a668325d2a7c admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:02.790884 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eeee14-a390-4a93-bbce-a668325d2a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:02.790884 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eeee14-a390-4a93-bbce-a668325d2a7c admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:02.792828 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eeee14-a390-4a93-bbce-a668325d2a7c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:18:02.792828 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eeee14-a390-4a93-bbce-a668325d2a7c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:02.795507 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eeee14-a390-4a93-bbce-a668325d2a7c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:02.796558 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05459d5b-b9a6-4e8f-9b42-28495875bfe6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:02.799677 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05459d5b-b9a6-4e8f-9b42-28495875bfe6 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:02.816669 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1066/2117] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 6 16:18:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:18:02.818777 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eeee14-a390-4a93-bbce-a668325d2a7c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:02.834976 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eeee14-a390-4a93-bbce-a668325d2a7c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:02.842039 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70eda86-1f83-4239-bfd3-12ccea4e664a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:02.842940 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70eda86-1f83-4239-bfd3-12ccea4e664a admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:02.843231 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70eda86-1f83-4239-bfd3-12ccea4e664a admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:02.843526 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70eda86-1f83-4239-bfd3-12ccea4e664a admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:02.844566 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70eda86-1f83-4239-bfd3-12ccea4e664a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:02.849605 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70eda86-1f83-4239-bfd3-12ccea4e664a admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:02.855761 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1052/2118] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:18:02 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:18:02.961156 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1067/2119] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:18:02 2022] POST /identity/v3/projects => generated 389 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:18:02.965944 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05459d5b-b9a6-4e8f-9b42-28495875bfe6 service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:02.987769 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05459d5b-b9a6-4e8f-9b42-28495875bfe6 service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:03.035861 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70eda86-1f83-4239-bfd3-12ccea4e664a admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:03.041216 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f70eda86-1f83-4239-bfd3-12ccea4e664a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:03.128860 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeb31e0-00e5-448a-b2ff-c9d1053abfb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:03.130352 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb31e0-00e5-448a-b2ff-c9d1053abfb4 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:03.131858 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb31e0-00e5-448a-b2ff-c9d1053abfb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:03.132600 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb31e0-00e5-448a-b2ff-c9d1053abfb4 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:03.133762 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeb31e0-00e5-448a-b2ff-c9d1053abfb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:03.137199 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeb31e0-00e5-448a-b2ff-c9d1053abfb4 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:03.151110 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70eda86-1f83-4239-bfd3-12ccea4e664a admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:03.160665 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf82f68-5f05-421e-9ec6-3460fb1ea480 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:03.161853 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf82f68-5f05-421e-9ec6-3460fb1ea480 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:03.162585 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf82f68-5f05-421e-9ec6-3460fb1ea480 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:03.163145 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf82f68-5f05-421e-9ec6-3460fb1ea480 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:03.167056 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-fdf82f68-5f05-421e-9ec6-3460fb1ea480 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:03.171783 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1053/2120] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:18:02 2022] POST /identity/v3/users => generated 295 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:18:03.179582 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeb31e0-00e5-448a-b2ff-c9d1053abfb4 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:03.202807 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeb31e0-00e5-448a-b2ff-c9d1053abfb4 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:03.212395 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809aed5b-5e91-4bbc-bf03-c9bca8ff497f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:03.213675 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809aed5b-5e91-4bbc-bf03-c9bca8ff497f admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:03.214296 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809aed5b-5e91-4bbc-bf03-c9bca8ff497f admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:03.214857 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809aed5b-5e91-4bbc-bf03-c9bca8ff497f admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:03.216176 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809aed5b-5e91-4bbc-bf03-c9bca8ff497f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:03.219081 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809aed5b-5e91-4bbc-bf03-c9bca8ff497f admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:03.240918 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1054/2121] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:18:03 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:18:03.242325 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809aed5b-5e91-4bbc-bf03-c9bca8ff497f admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:03.260086 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809aed5b-5e91-4bbc-bf03-c9bca8ff497f admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:03.267286 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3136f151-b02d-44eb-9935-d39e834b776a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:03.270006 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3136f151-b02d-44eb-9935-d39e834b776a admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:03.270548 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3136f151-b02d-44eb-9935-d39e834b776a admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:03.271149 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3136f151-b02d-44eb-9935-d39e834b776a admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:03.272061 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3136f151-b02d-44eb-9935-d39e834b776a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:03.274437 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3136f151-b02d-44eb-9935-d39e834b776a admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:03.293529 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1055/2122] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:18:03 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:18:03.294931 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3136f151-b02d-44eb-9935-d39e834b776a admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:03.304390 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf82f68-5f05-421e-9ec6-3460fb1ea480 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:03.307831 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf82f68-5f05-421e-9ec6-3460fb1ea480 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:03.311434 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3136f151-b02d-44eb-9935-d39e834b776a admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:03.320479 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e39ee5e-ecd4-4423-a71d-010b8f56d472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:03.321037 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:18:03.325345 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e39ee5e-ecd4-4423-a71d-010b8f56d472 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:03.326046 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e39ee5e-ecd4-4423-a71d-010b8f56d472 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:03.326641 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e39ee5e-ecd4-4423-a71d-010b8f56d472 admin admin] PATH_INFO: `/v3/projects/4fec6aa2d0c54645a84c040a7ae46ce4/users/247a863de2514d7c87f3e7c8fab5b428/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:03.355769 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e39ee5e-ecd4-4423-a71d-010b8f56d472 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fec6aa2d0c54645a84c040a7ae46ce4, user_id=247a863de2514d7c87f3e7c8fab5b428, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:03.356338 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1068/2123] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:18:03 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:18:03.361557 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e39ee5e-ecd4-4423-a71d-010b8f56d472 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:03.425945 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1056/2124] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:18:03 2022] PUT /identity/v3/projects/4fec6aa2d0c54645a84c040a7ae46ce4/users/247a863de2514d7c87f3e7c8fab5b428/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:18:03.429791 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3126e2-cace-4542-b17e-af62315f4422 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:03.430379 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3126e2-cace-4542-b17e-af62315f4422 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:03.430840 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3126e2-cace-4542-b17e-af62315f4422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:03.516192 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-8b3126e2-cace-4542-b17e-af62315f4422 None None] MFA Rules not processed for user `247a863de2514d7c87f3e7c8fab5b428`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:18:03.517895 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-8b3126e2-cace-4542-b17e-af62315f4422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:03.642295 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1069/2125] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:18:03 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:18:03.651632 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e39ee5e-ecd4-4423-a71d-010b8f56d472 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:03.704079 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:18:03.743236 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e39ee5e-ecd4-4423-a71d-010b8f56d472 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:03.756278 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fb9e29-a0a4-42bd-931c-a079aa7c63a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:03.757111 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb9e29-a0a4-42bd-931c-a079aa7c63a4 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:03.757434 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb9e29-a0a4-42bd-931c-a079aa7c63a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:03.757763 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb9e29-a0a4-42bd-931c-a079aa7c63a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:03.767267 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fb9e29-a0a4-42bd-931c-a079aa7c63a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:03.770270 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fb9e29-a0a4-42bd-931c-a079aa7c63a4 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:03.782858 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1057/2126] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:18:03 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:18:05.751704 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f84c97-7a78-4cd5-9132-f2a3fbc16998 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:05.752349 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f84c97-7a78-4cd5-9132-f2a3fbc16998 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:05.752349 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f84c97-7a78-4cd5-9132-f2a3fbc16998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:05.801972 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-70f84c97-7a78-4cd5-9132-f2a3fbc16998 None None] MFA Rules not processed for user `a280595c6ff4495ab9f4120b4bcc60c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:18:05.803454 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-70f84c97-7a78-4cd5-9132-f2a3fbc16998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:05.905035 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1070/2127] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:18:05 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:18:05.909959 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fb9e29-a0a4-42bd-931c-a079aa7c63a4 service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:05.911495 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-83fb9e29-a0a4-42bd-931c-a079aa7c63a4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:06.001546 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fb9e29-a0a4-42bd-931c-a079aa7c63a4 service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:06.007171 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf11e6d4-9cfb-4847-a3c7-8066de7b1197 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c163b39e61648c385f065ce8091c3ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:06.007981 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf11e6d4-9cfb-4847-a3c7-8066de7b1197 service glance] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:06.008366 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf11e6d4-9cfb-4847-a3c7-8066de7b1197 service glance] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:06.008801 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf11e6d4-9cfb-4847-a3c7-8066de7b1197 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:06.011444 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-cf11e6d4-9cfb-4847-a3c7-8066de7b1197 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:06.139238 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf11e6d4-9cfb-4847-a3c7-8066de7b1197 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:06.141860 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf11e6d4-9cfb-4847-a3c7-8066de7b1197 service glance] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:06.155824 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1058/2128] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:18:05 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:18:06.380860 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc41c94-03d0-4503-8a4d-97f759b96b15 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:06.381164 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc41c94-03d0-4503-8a4d-97f759b96b15 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:06.381418 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc41c94-03d0-4503-8a4d-97f759b96b15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:06.429617 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-4dc41c94-03d0-4503-8a4d-97f759b96b15 None None] MFA Rules not processed for user `d865006413b24ec3922619311f81f353`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:18:06.431658 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-4dc41c94-03d0-4503-8a4d-97f759b96b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:06.568133 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1071/2129] 10.222.0.43 () {58 vars in 1009 bytes} [Tue Sep 6 16:18:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:18:06.645101 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf11e6d4-9cfb-4847-a3c7-8066de7b1197 service glance] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:06.647413 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-cf11e6d4-9cfb-4847-a3c7-8066de7b1197 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:06.770038 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf11e6d4-9cfb-4847-a3c7-8066de7b1197 service glance] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:06.775696 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb17ba9d-8d18-4f78-be68-9df4eb7cb1b7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '312026c080b64aff9dd483d3591288a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:06.776317 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb17ba9d-8d18-4f78-be68-9df4eb7cb1b7 service swift] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:06.776743 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb17ba9d-8d18-4f78-be68-9df4eb7cb1b7 service swift] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:06.777048 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb17ba9d-8d18-4f78-be68-9df4eb7cb1b7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:06.786361 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb17ba9d-8d18-4f78-be68-9df4eb7cb1b7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:06.788084 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb17ba9d-8d18-4f78-be68-9df4eb7cb1b7 service swift] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:06.805444 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1059/2130] 10.222.0.43 () {60 vars in 1384 bytes} [Tue Sep 6 16:18:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 16:18:16.549213 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=92 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:18:16.794094 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=91 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:18:17.197007 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f653e232-bc97-4b9f-825f-f1d82e300081 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:17.197295 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f653e232-bc97-4b9f-825f-f1d82e300081 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:17.197571 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f653e232-bc97-4b9f-825f-f1d82e300081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:17.219780 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-f653e232-bc97-4b9f-825f-f1d82e300081 None None] MFA Rules not processed for user `bbe4c1ecf08f4dd5a005f883c16c6a62`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:18:17.221070 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f653e232-bc97-4b9f-825f-f1d82e300081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:17.228269 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1072/2131] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:18:17 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:18:19.140937 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb17ba9d-8d18-4f78-be68-9df4eb7cb1b7 service swift] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:19.142470 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-bb17ba9d-8d18-4f78-be68-9df4eb7cb1b7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:19.209540 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb17ba9d-8d18-4f78-be68-9df4eb7cb1b7 service swift] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:19.214519 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921c1a93-f0cd-4c25-a8d5-f18400e444b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:19.215155 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921c1a93-f0cd-4c25-a8d5-f18400e444b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:19.215430 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921c1a93-f0cd-4c25-a8d5-f18400e444b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:19.215705 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921c1a93-f0cd-4c25-a8d5-f18400e444b8 admin admin] PATH_INFO: `/v3/users/2a5bec8038c64151af84cb1c9d5cfd35` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:19.216888 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921c1a93-f0cd-4c25-a8d5-f18400e444b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a5bec8038c64151af84cb1c9d5cfd35)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:19.218871 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921c1a93-f0cd-4c25-a8d5-f18400e444b8 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:19.268647 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-921c1a93-f0cd-4c25-a8d5-f18400e444b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a5bec8038c64151af84cb1c9d5cfd35'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:19.269375 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-921c1a93-f0cd-4c25-a8d5-f18400e444b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a5bec8038c64151af84cb1c9d5cfd35'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:19.276819 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-921c1a93-f0cd-4c25-a8d5-f18400e444b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a5bec8038c64151af84cb1c9d5cfd35'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:19.289288 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-921c1a93-f0cd-4c25-a8d5-f18400e444b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a5bec8038c64151af84cb1c9d5cfd35'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:19.298373 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-921c1a93-f0cd-4c25-a8d5-f18400e444b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a5bec8038c64151af84cb1c9d5cfd35'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:19.304488 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1060/2132] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:18:19 2022] DELETE /identity/v3/users/2a5bec8038c64151af84cb1c9d5cfd35 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:18:19.309261 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b517d7-670b-4649-a087-c077ea3e090d None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:19.309786 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b517d7-670b-4649-a087-c077ea3e090d None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:19.309869 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b517d7-670b-4649-a087-c077ea3e090d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:19.329616 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-10b517d7-670b-4649-a087-c077ea3e090d None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:18:19.330914 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-10b517d7-670b-4649-a087-c077ea3e090d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:19.398277 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1073/2133] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:18:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:18:19.409274 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921c1a93-f0cd-4c25-a8d5-f18400e444b8 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:19.410801 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-921c1a93-f0cd-4c25-a8d5-f18400e444b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:19.473658 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921c1a93-f0cd-4c25-a8d5-f18400e444b8 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:19.478353 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e86128-44ed-4f3d-a701-54dc165141e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:19.479015 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e86128-44ed-4f3d-a701-54dc165141e9 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:19.479290 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e86128-44ed-4f3d-a701-54dc165141e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:19.479962 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e86128-44ed-4f3d-a701-54dc165141e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:19.494820 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e86128-44ed-4f3d-a701-54dc165141e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:19.496836 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e86128-44ed-4f3d-a701-54dc165141e9 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:19.512399 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1061/2134] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:18:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:18:19.863502 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b517d7-670b-4649-a087-c077ea3e090d None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:19.865007 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-10b517d7-670b-4649-a087-c077ea3e090d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:19.879304 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b517d7-670b-4649-a087-c077ea3e090d None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:19.891683 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469ac12a-6fcf-4d29-a0ab-29c2bf11292a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:19.892340 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ac12a-6fcf-4d29-a0ab-29c2bf11292a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:19.892618 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ac12a-6fcf-4d29-a0ab-29c2bf11292a admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:19.892924 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ac12a-6fcf-4d29-a0ab-29c2bf11292a admin admin] PATH_INFO: `/v3/projects/53173c7ff08343adace7682eb5997f94` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:19.894637 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469ac12a-6fcf-4d29-a0ab-29c2bf11292a admin admin] RBAC: Authorizing `identity:delete_project(project_id=53173c7ff08343adace7682eb5997f94)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:19.896626 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469ac12a-6fcf-4d29-a0ab-29c2bf11292a admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:19.968786 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-469ac12a-6fcf-4d29-a0ab-29c2bf11292a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53173c7ff08343adace7682eb5997f94', 'request_id': 'req-469ac12a-6fcf-4d29-a0ab-29c2bf11292a'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:19.968786 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-469ac12a-6fcf-4d29-a0ab-29c2bf11292a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53173c7ff08343adace7682eb5997f94', 'request_id': 'req-469ac12a-6fcf-4d29-a0ab-29c2bf11292a'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:19.979045 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-469ac12a-6fcf-4d29-a0ab-29c2bf11292a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53173c7ff08343adace7682eb5997f94', 'request_id': 'req-469ac12a-6fcf-4d29-a0ab-29c2bf11292a'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:19.995221 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-469ac12a-6fcf-4d29-a0ab-29c2bf11292a admin admin] The token cache is being invalidate because project 53173c7ff08343adace7682eb5997f94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:18:19.995494 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-469ac12a-6fcf-4d29-a0ab-29c2bf11292a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:19.996158 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-469ac12a-6fcf-4d29-a0ab-29c2bf11292a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:19.997735 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1074/2135] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:18:19 2022] DELETE /identity/v3/projects/53173c7ff08343adace7682eb5997f94 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:18:20.005880 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b238432-a89e-4ff3-bcd6-87cd81203553 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:20.006165 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b238432-a89e-4ff3-bcd6-87cd81203553 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:20.006474 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b238432-a89e-4ff3-bcd6-87cd81203553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:20.027125 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-1b238432-a89e-4ff3-bcd6-87cd81203553 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:18:20.028362 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-1b238432-a89e-4ff3-bcd6-87cd81203553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:20.046620 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:18:20.094175 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1062/2136] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:18:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:18:20.103598 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afccf8c1-610b-429c-ae38-e9a574add706 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:20.103951 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afccf8c1-610b-429c-ae38-e9a574add706 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:20.104233 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afccf8c1-610b-429c-ae38-e9a574add706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:20.125728 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-afccf8c1-610b-429c-ae38-e9a574add706 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:18:20.126964 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-afccf8c1-610b-429c-ae38-e9a574add706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:20.135182 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1075/2137] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:18:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:18:20.139699 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b238432-a89e-4ff3-bcd6-87cd81203553 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:20.149808 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b238432-a89e-4ff3-bcd6-87cd81203553 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:20.154797 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaf3292-2f46-4afd-ba99-3f3657ef974c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:20.155423 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf3292-2f46-4afd-ba99-3f3657ef974c admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:20.155698 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf3292-2f46-4afd-ba99-3f3657ef974c admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:20.155988 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf3292-2f46-4afd-ba99-3f3657ef974c admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:20.156647 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaf3292-2f46-4afd-ba99-3f3657ef974c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:18:20.156990 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaf3292-2f46-4afd-ba99-3f3657ef974c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:20.158816 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaf3292-2f46-4afd-ba99-3f3657ef974c admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:20.171713 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1063/2138] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 6 16:18:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:18:20.176465 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afccf8c1-610b-429c-ae38-e9a574add706 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:20.186319 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afccf8c1-610b-429c-ae38-e9a574add706 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:20.191619 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d51bd4a-d90a-4011-bcdf-48cb759ab0e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:20.192298 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d51bd4a-d90a-4011-bcdf-48cb759ab0e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:20.192648 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d51bd4a-d90a-4011-bcdf-48cb759ab0e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:20.193074 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d51bd4a-d90a-4011-bcdf-48cb759ab0e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:20.193775 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d51bd4a-d90a-4011-bcdf-48cb759ab0e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:20.195944 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d51bd4a-d90a-4011-bcdf-48cb759ab0e3 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:20.227649 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1076/2139] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:18:20 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:18:20.233291 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaf3292-2f46-4afd-ba99-3f3657ef974c admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:20.244312 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaf3292-2f46-4afd-ba99-3f3657ef974c admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:20.305943 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05ec0cf-76d7-4c29-bfe7-ad7157a3d522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:20.306473 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05ec0cf-76d7-4c29-bfe7-ad7157a3d522 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:20.306755 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05ec0cf-76d7-4c29-bfe7-ad7157a3d522 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:20.307030 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05ec0cf-76d7-4c29-bfe7-ad7157a3d522 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:20.307692 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05ec0cf-76d7-4c29-bfe7-ad7157a3d522 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:20.309688 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05ec0cf-76d7-4c29-bfe7-ad7157a3d522 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:20.340665 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1064/2140] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:18:20 2022] POST /identity/v3/users => generated 293 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:18:20.344961 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d51bd4a-d90a-4011-bcdf-48cb759ab0e3 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:20.356281 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d51bd4a-d90a-4011-bcdf-48cb759ab0e3 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:20.362253 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97d5973-724f-4257-bbf9-462a36b60192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:20.362868 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d5973-724f-4257-bbf9-462a36b60192 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:20.363141 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d5973-724f-4257-bbf9-462a36b60192 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:20.363404 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d5973-724f-4257-bbf9-462a36b60192 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:20.364047 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97d5973-724f-4257-bbf9-462a36b60192 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:20.365705 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97d5973-724f-4257-bbf9-462a36b60192 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:20.376583 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1077/2141] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:18:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 16:18:20.381186 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05ec0cf-76d7-4c29-bfe7-ad7157a3d522 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:20.391838 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05ec0cf-76d7-4c29-bfe7-ad7157a3d522 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:20.397623 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5437be-064b-4a34-a37e-a30c30ce36bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:20.398241 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5437be-064b-4a34-a37e-a30c30ce36bc admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:20.398513 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5437be-064b-4a34-a37e-a30c30ce36bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:20.398785 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5437be-064b-4a34-a37e-a30c30ce36bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:20.399429 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5437be-064b-4a34-a37e-a30c30ce36bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:20.401094 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5437be-064b-4a34-a37e-a30c30ce36bc admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:20.413130 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1065/2142] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:18:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:18:20.417511 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97d5973-724f-4257-bbf9-462a36b60192 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:20.429063 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97d5973-724f-4257-bbf9-462a36b60192 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:20.436103 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb44c63d-397f-4aaf-b4cc-2e42d80c40d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:20.436951 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb44c63d-397f-4aaf-b4cc-2e42d80c40d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:20.437229 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb44c63d-397f-4aaf-b4cc-2e42d80c40d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:20.437504 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb44c63d-397f-4aaf-b4cc-2e42d80c40d3 admin admin] PATH_INFO: `/v3/projects/7b7d42cddc04425c99641db191f56de6/users/1af83dff7a9e4e6d9e8bc15223a54096/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:20.453534 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb44c63d-397f-4aaf-b4cc-2e42d80c40d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b7d42cddc04425c99641db191f56de6, user_id=1af83dff7a9e4e6d9e8bc15223a54096, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:20.456214 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb44c63d-397f-4aaf-b4cc-2e42d80c40d3 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:20.473407 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1078/2143] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:18:20 2022] PUT /identity/v3/projects/7b7d42cddc04425c99641db191f56de6/users/1af83dff7a9e4e6d9e8bc15223a54096/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:18:20.484340 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df5c3e-b2b6-4b9e-9260-255e172f913d None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:20.484601 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df5c3e-b2b6-4b9e-9260-255e172f913d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:20.484953 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df5c3e-b2b6-4b9e-9260-255e172f913d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:20.521759 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-54df5c3e-b2b6-4b9e-9260-255e172f913d None None] MFA Rules not processed for user `1af83dff7a9e4e6d9e8bc15223a54096`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:18:20.523039 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-54df5c3e-b2b6-4b9e-9260-255e172f913d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:20.594665 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1066/2144] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:18:20 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:18:20.599725 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb44c63d-397f-4aaf-b4cc-2e42d80c40d3 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:20.616542 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:18:20.675237 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb44c63d-397f-4aaf-b4cc-2e42d80c40d3 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:20.680933 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef6483a-cda2-417e-a3de-a54c66bb4603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:20.681756 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef6483a-cda2-417e-a3de-a54c66bb4603 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:20.682034 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef6483a-cda2-417e-a3de-a54c66bb4603 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:20.682304 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef6483a-cda2-417e-a3de-a54c66bb4603 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:20.682979 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef6483a-cda2-417e-a3de-a54c66bb4603 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:20.685054 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef6483a-cda2-417e-a3de-a54c66bb4603 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:20.718493 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1079/2145] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:18:20 2022] POST /identity/v3/projects => generated 383 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:18:20.719678 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54df5c3e-b2b6-4b9e-9260-255e172f913d None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:20.731000 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54df5c3e-b2b6-4b9e-9260-255e172f913d None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:20.745443 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:18:20.812891 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e305fbd-67c8-4fb7-9765-70b8328d01d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:20.813777 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e305fbd-67c8-4fb7-9765-70b8328d01d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:20.814065 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e305fbd-67c8-4fb7-9765-70b8328d01d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:20.814333 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e305fbd-67c8-4fb7-9765-70b8328d01d2 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:20.815013 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e305fbd-67c8-4fb7-9765-70b8328d01d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:20.816996 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e305fbd-67c8-4fb7-9765-70b8328d01d2 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:20.839090 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1067/2146] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:18:20 2022] POST /identity/v3/users => generated 292 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:18:20.843511 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef6483a-cda2-417e-a3de-a54c66bb4603 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:20.853964 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef6483a-cda2-417e-a3de-a54c66bb4603 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:20.858544 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b13bfa-db74-448d-a4e5-c1d45dfcbdea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:20.859152 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b13bfa-db74-448d-a4e5-c1d45dfcbdea admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:20.859427 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b13bfa-db74-448d-a4e5-c1d45dfcbdea admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:20.859691 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b13bfa-db74-448d-a4e5-c1d45dfcbdea admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:20.860334 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b13bfa-db74-448d-a4e5-c1d45dfcbdea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:20.862110 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b13bfa-db74-448d-a4e5-c1d45dfcbdea admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:20.873335 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1080/2147] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:18:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:18:20.877538 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e305fbd-67c8-4fb7-9765-70b8328d01d2 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:20.888098 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e305fbd-67c8-4fb7-9765-70b8328d01d2 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:20.894653 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717fde29-fef4-47ea-bd66-76a94945addf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:20.895278 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717fde29-fef4-47ea-bd66-76a94945addf admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:20.895534 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717fde29-fef4-47ea-bd66-76a94945addf admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:20.895801 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717fde29-fef4-47ea-bd66-76a94945addf admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:20.896636 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717fde29-fef4-47ea-bd66-76a94945addf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:20.898288 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717fde29-fef4-47ea-bd66-76a94945addf admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:20.909534 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1068/2148] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:18:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:18:20.915278 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b13bfa-db74-448d-a4e5-c1d45dfcbdea admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:20.929220 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b13bfa-db74-448d-a4e5-c1d45dfcbdea admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:20.936821 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da823ba0-7d36-4e16-924f-0512dd7c5cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:20.937458 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da823ba0-7d36-4e16-924f-0512dd7c5cfc admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:20.937736 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da823ba0-7d36-4e16-924f-0512dd7c5cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:20.938012 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da823ba0-7d36-4e16-924f-0512dd7c5cfc admin admin] PATH_INFO: `/v3/projects/b93250123e1344b78a9c9898a3dd7b4b/users/c2d5b77c08774470816e3ddf2bb3be2c/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:20.953565 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da823ba0-7d36-4e16-924f-0512dd7c5cfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=b93250123e1344b78a9c9898a3dd7b4b, user_id=c2d5b77c08774470816e3ddf2bb3be2c, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:20.956330 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da823ba0-7d36-4e16-924f-0512dd7c5cfc admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:20.970423 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1081/2149] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:18:20 2022] PUT /identity/v3/projects/b93250123e1344b78a9c9898a3dd7b4b/users/c2d5b77c08774470816e3ddf2bb3be2c/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:18:20.981542 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877dcc07-4089-4c1d-9c57-a95a707ac41d None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:20.983125 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877dcc07-4089-4c1d-9c57-a95a707ac41d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:20.983429 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877dcc07-4089-4c1d-9c57-a95a707ac41d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:21.030606 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-877dcc07-4089-4c1d-9c57-a95a707ac41d None None] MFA Rules not processed for user `c2d5b77c08774470816e3ddf2bb3be2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:18:21.031823 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-877dcc07-4089-4c1d-9c57-a95a707ac41d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:21.141342 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1069/2150] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:18:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:18:21.141432 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da823ba0-7d36-4e16-924f-0512dd7c5cfc admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:21.216026 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da823ba0-7d36-4e16-924f-0512dd7c5cfc admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:21.221410 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc21df04-5aaa-4160-9940-b9639c8314a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:21.222184 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc21df04-5aaa-4160-9940-b9639c8314a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:21.222606 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc21df04-5aaa-4160-9940-b9639c8314a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:21.222845 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc21df04-5aaa-4160-9940-b9639c8314a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:21.223575 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc21df04-5aaa-4160-9940-b9639c8314a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:21.225632 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc21df04-5aaa-4160-9940-b9639c8314a5 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:21.264022 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1082/2151] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:18:21 2022] POST /identity/v3/projects => generated 385 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:18:21.268357 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877dcc07-4089-4c1d-9c57-a95a707ac41d None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:21.284164 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877dcc07-4089-4c1d-9c57-a95a707ac41d None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:21.352242 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74282eb5-38e3-4ad0-9c27-234bb9ad0247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:21.352871 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74282eb5-38e3-4ad0-9c27-234bb9ad0247 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:21.353149 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74282eb5-38e3-4ad0-9c27-234bb9ad0247 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:21.353457 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74282eb5-38e3-4ad0-9c27-234bb9ad0247 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:21.354161 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74282eb5-38e3-4ad0-9c27-234bb9ad0247 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:21.356096 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74282eb5-38e3-4ad0-9c27-234bb9ad0247 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:21.379915 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1070/2152] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:18:21 2022] POST /identity/v3/users => generated 299 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:18:21.384242 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc21df04-5aaa-4160-9940-b9639c8314a5 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:21.396143 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc21df04-5aaa-4160-9940-b9639c8314a5 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:21.401016 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47a8802-957d-49a5-a2a8-20840e890312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:21.401620 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a8802-957d-49a5-a2a8-20840e890312 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:21.401908 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a8802-957d-49a5-a2a8-20840e890312 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:21.402159 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a8802-957d-49a5-a2a8-20840e890312 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:21.402828 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47a8802-957d-49a5-a2a8-20840e890312 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:21.404437 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47a8802-957d-49a5-a2a8-20840e890312 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:21.417579 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1083/2153] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:18:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:18:21.424892 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74282eb5-38e3-4ad0-9c27-234bb9ad0247 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:21.436302 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74282eb5-38e3-4ad0-9c27-234bb9ad0247 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:21.441219 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010c50c3-5276-4d6d-982a-73023cfc7b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:21.441888 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c50c3-5276-4d6d-982a-73023cfc7b28 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:21.442160 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c50c3-5276-4d6d-982a-73023cfc7b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:21.442431 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c50c3-5276-4d6d-982a-73023cfc7b28 admin admin] PATH_INFO: `/v3/projects/5e48edaea09c469c96ae6baadce14a9b/users/2c14dd1fe11641449b4c227f6c9c171f/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:21.455564 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:18:21.461432 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010c50c3-5276-4d6d-982a-73023cfc7b28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e48edaea09c469c96ae6baadce14a9b, user_id=2c14dd1fe11641449b4c227f6c9c171f, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:21.466887 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010c50c3-5276-4d6d-982a-73023cfc7b28 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:21.488486 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1071/2154] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:18:21 2022] PUT /identity/v3/projects/5e48edaea09c469c96ae6baadce14a9b/users/2c14dd1fe11641449b4c227f6c9c171f/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:18:21.492896 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47a8802-957d-49a5-a2a8-20840e890312 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:21.503727 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47a8802-957d-49a5-a2a8-20840e890312 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:21.538216 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:18:21.564031 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af32257-924f-436f-8139-d8a1919c7bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:21.564647 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af32257-924f-436f-8139-d8a1919c7bce admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:21.564964 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af32257-924f-436f-8139-d8a1919c7bce admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:21.565243 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af32257-924f-436f-8139-d8a1919c7bce admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:21.565899 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af32257-924f-436f-8139-d8a1919c7bce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:21.567535 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af32257-924f-436f-8139-d8a1919c7bce admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:21.577954 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1084/2155] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:18:21 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:18:21.583412 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010c50c3-5276-4d6d-982a-73023cfc7b28 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:21.599115 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010c50c3-5276-4d6d-982a-73023cfc7b28 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:21.604605 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b40f4b-0056-4e77-b900-8e8940b86be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:21.605237 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b40f4b-0056-4e77-b900-8e8940b86be0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:21.605555 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b40f4b-0056-4e77-b900-8e8940b86be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:21.605820 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b40f4b-0056-4e77-b900-8e8940b86be0 admin admin] PATH_INFO: `/v3/projects/5e48edaea09c469c96ae6baadce14a9b/users/2c14dd1fe11641449b4c227f6c9c171f/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:21.608024 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b40f4b-0056-4e77-b900-8e8940b86be0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e48edaea09c469c96ae6baadce14a9b, user_id=2c14dd1fe11641449b4c227f6c9c171f, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:21.610740 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b40f4b-0056-4e77-b900-8e8940b86be0 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:21.623537 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1072/2156] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:18:21 2022] PUT /identity/v3/projects/5e48edaea09c469c96ae6baadce14a9b/users/2c14dd1fe11641449b4c227f6c9c171f/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:18:21.635470 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84347b3d-e1bd-44ae-922f-428803d5ec0e None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:21.635821 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84347b3d-e1bd-44ae-922f-428803d5ec0e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:21.636079 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84347b3d-e1bd-44ae-922f-428803d5ec0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:21.666389 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-84347b3d-e1bd-44ae-922f-428803d5ec0e None None] MFA Rules not processed for user `2c14dd1fe11641449b4c227f6c9c171f`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:18:21.667607 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-84347b3d-e1bd-44ae-922f-428803d5ec0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:21.736567 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1085/2157] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:18:21 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:18:21.750472 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b40f4b-0056-4e77-b900-8e8940b86be0 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:21.752710 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c6b40f4b-0056-4e77-b900-8e8940b86be0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:21.829253 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b40f4b-0056-4e77-b900-8e8940b86be0 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:21.833918 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ec50b6-139e-41fd-95e3-0d52f1488274 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:21.834588 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ec50b6-139e-41fd-95e3-0d52f1488274 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:21.834927 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ec50b6-139e-41fd-95e3-0d52f1488274 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:21.835253 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ec50b6-139e-41fd-95e3-0d52f1488274 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:21.904283 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ec50b6-139e-41fd-95e3-0d52f1488274 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:21.907113 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ec50b6-139e-41fd-95e3-0d52f1488274 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:21.920529 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1073/2158] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:18:21 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:18:26.710261 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84347b3d-e1bd-44ae-922f-428803d5ec0e None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:26.728008 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84347b3d-e1bd-44ae-922f-428803d5ec0e None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:26.732428 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736180f8-e62f-4a77-ab76-13687cfe93f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:26.733220 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736180f8-e62f-4a77-ab76-13687cfe93f0 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:26.733498 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736180f8-e62f-4a77-ab76-13687cfe93f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:26.733828 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736180f8-e62f-4a77-ab76-13687cfe93f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:26.820160 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736180f8-e62f-4a77-ab76-13687cfe93f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:26.820929 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736180f8-e62f-4a77-ab76-13687cfe93f0 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:26.831931 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1086/2159] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:18:26 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:18:27.597655 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ec50b6-139e-41fd-95e3-0d52f1488274 service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:27.606590 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ec50b6-139e-41fd-95e3-0d52f1488274 service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:27.611247 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefe3153-29d2-4fd1-a01c-a56852b0e9e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:27.612079 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe3153-29d2-4fd1-a01c-a56852b0e9e4 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:27.612383 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe3153-29d2-4fd1-a01c-a56852b0e9e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:27.612639 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe3153-29d2-4fd1-a01c-a56852b0e9e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:27.622963 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefe3153-29d2-4fd1-a01c-a56852b0e9e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:27.624518 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefe3153-29d2-4fd1-a01c-a56852b0e9e4 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:27.635166 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1074/2160] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:18:27 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:18:30.001939 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51b6802-3b5e-4024-9fc2-d0d53adf636f None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:30.002555 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51b6802-3b5e-4024-9fc2-d0d53adf636f None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:30.002911 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51b6802-3b5e-4024-9fc2-d0d53adf636f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:30.041755 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-c51b6802-3b5e-4024-9fc2-d0d53adf636f None None] MFA Rules not processed for user `d865006413b24ec3922619311f81f353`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:18:30.043920 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-c51b6802-3b5e-4024-9fc2-d0d53adf636f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:30.139784 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1087/2161] 10.222.0.43 () {58 vars in 1009 bytes} [Tue Sep 6 16:18:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:18:30.161822 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefe3153-29d2-4fd1-a01c-a56852b0e9e4 service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:30.163933 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-aefe3153-29d2-4fd1-a01c-a56852b0e9e4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:30.290799 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefe3153-29d2-4fd1-a01c-a56852b0e9e4 service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:30.296895 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561d8253-e6aa-4e3e-84c4-e3301a3151da service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '312026c080b64aff9dd483d3591288a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:30.297576 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561d8253-e6aa-4e3e-84c4-e3301a3151da service swift] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:30.297862 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561d8253-e6aa-4e3e-84c4-e3301a3151da service swift] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:30.298137 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561d8253-e6aa-4e3e-84c4-e3301a3151da service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:30.309599 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561d8253-e6aa-4e3e-84c4-e3301a3151da service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:30.311688 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561d8253-e6aa-4e3e-84c4-e3301a3151da service swift] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:30.322390 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1075/2162] 10.222.0.43 () {60 vars in 1384 bytes} [Tue Sep 6 16:18:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 06 16:18:30.639576 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51b6802-3b5e-4024-9fc2-d0d53adf636f None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:30.641851 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-c51b6802-3b5e-4024-9fc2-d0d53adf636f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:30.649352 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:18:30.756969 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51b6802-3b5e-4024-9fc2-d0d53adf636f None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:30.766176 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e88952e-1e2d-4171-baf2-d221654eec6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:30.767395 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e88952e-1e2d-4171-baf2-d221654eec6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:30.767937 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e88952e-1e2d-4171-baf2-d221654eec6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:30.768405 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e88952e-1e2d-4171-baf2-d221654eec6d admin admin] PATH_INFO: `/v3/users/247a863de2514d7c87f3e7c8fab5b428` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:30.770051 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e88952e-1e2d-4171-baf2-d221654eec6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=247a863de2514d7c87f3e7c8fab5b428)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:30.772864 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e88952e-1e2d-4171-baf2-d221654eec6d admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:30.871275 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8e88952e-1e2d-4171-baf2-d221654eec6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '247a863de2514d7c87f3e7c8fab5b428'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:30.873794 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8e88952e-1e2d-4171-baf2-d221654eec6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '247a863de2514d7c87f3e7c8fab5b428'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:30.903890 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8e88952e-1e2d-4171-baf2-d221654eec6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '247a863de2514d7c87f3e7c8fab5b428'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:30.933067 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8e88952e-1e2d-4171-baf2-d221654eec6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '247a863de2514d7c87f3e7c8fab5b428'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:30.935423 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8e88952e-1e2d-4171-baf2-d221654eec6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '247a863de2514d7c87f3e7c8fab5b428'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:30.981439 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1088/2163] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:18:30 2022] DELETE /identity/v3/users/247a863de2514d7c87f3e7c8fab5b428 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:18:30.983923 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e2a2f-60d7-4f8b-b14a-1c41f30a919c None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:30.984197 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e2a2f-60d7-4f8b-b14a-1c41f30a919c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:30.985199 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e2a2f-60d7-4f8b-b14a-1c41f30a919c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:31.045984 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-a33e2a2f-60d7-4f8b-b14a-1c41f30a919c None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:18:31.060976 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a33e2a2f-60d7-4f8b-b14a-1c41f30a919c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:31.074315 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:18:31.276162 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1076/2164] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:18:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:18:31.294693 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e88952e-1e2d-4171-baf2-d221654eec6d admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:31.296495 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-8e88952e-1e2d-4171-baf2-d221654eec6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:31.545505 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e88952e-1e2d-4171-baf2-d221654eec6d admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:31.576116 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d098126-007d-4a71-a022-ba98b922d5b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:31.577574 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d098126-007d-4a71-a022-ba98b922d5b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:31.584745 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d098126-007d-4a71-a022-ba98b922d5b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:31.587187 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d098126-007d-4a71-a022-ba98b922d5b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:31.644312 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d098126-007d-4a71-a022-ba98b922d5b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:31.654140 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d098126-007d-4a71-a022-ba98b922d5b7 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:31.704935 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1089/2165] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:18:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:18:32.357680 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33e2a2f-60d7-4f8b-b14a-1c41f30a919c None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:32.360141 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a33e2a2f-60d7-4f8b-b14a-1c41f30a919c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:32.430555 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33e2a2f-60d7-4f8b-b14a-1c41f30a919c None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:32.459183 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a7f732-5fb7-4dc5-b7ef-1256ad385516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:32.460995 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7f732-5fb7-4dc5-b7ef-1256ad385516 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:32.461771 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7f732-5fb7-4dc5-b7ef-1256ad385516 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:32.462508 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7f732-5fb7-4dc5-b7ef-1256ad385516 admin admin] PATH_INFO: `/v3/users/1af83dff7a9e4e6d9e8bc15223a54096` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:32.465233 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a7f732-5fb7-4dc5-b7ef-1256ad385516 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1af83dff7a9e4e6d9e8bc15223a54096)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:32.469002 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a7f732-5fb7-4dc5-b7ef-1256ad385516 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:32.506646 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d098126-007d-4a71-a022-ba98b922d5b7 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:32.508262 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-8d098126-007d-4a71-a022-ba98b922d5b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:32.533139 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d098126-007d-4a71-a022-ba98b922d5b7 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:32.548594 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a5ed2e-662b-42ba-8227-5169ccab87ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:32.549294 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a5ed2e-662b-42ba-8227-5169ccab87ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:32.549625 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a5ed2e-662b-42ba-8227-5169ccab87ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:32.549922 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a5ed2e-662b-42ba-8227-5169ccab87ed admin admin] PATH_INFO: `/v3/projects/4fec6aa2d0c54645a84c040a7ae46ce4` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:32.553813 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a5ed2e-662b-42ba-8227-5169ccab87ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fec6aa2d0c54645a84c040a7ae46ce4)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:32.555857 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a5ed2e-662b-42ba-8227-5169ccab87ed admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:32.592698 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:18:32.660884 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e8a7f732-5fb7-4dc5-b7ef-1256ad385516 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1af83dff7a9e4e6d9e8bc15223a54096'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:32.665419 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e8a7f732-5fb7-4dc5-b7ef-1256ad385516 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1af83dff7a9e4e6d9e8bc15223a54096'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:32.675759 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e7a5ed2e-662b-42ba-8227-5169ccab87ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fec6aa2d0c54645a84c040a7ae46ce4', 'request_id': 'req-e7a5ed2e-662b-42ba-8227-5169ccab87ed'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:32.696492 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e7a5ed2e-662b-42ba-8227-5169ccab87ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fec6aa2d0c54645a84c040a7ae46ce4', 'request_id': 'req-e7a5ed2e-662b-42ba-8227-5169ccab87ed'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:32.707415 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e8a7f732-5fb7-4dc5-b7ef-1256ad385516 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1af83dff7a9e4e6d9e8bc15223a54096'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:32.751269 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e7a5ed2e-662b-42ba-8227-5169ccab87ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fec6aa2d0c54645a84c040a7ae46ce4', 'request_id': 'req-e7a5ed2e-662b-42ba-8227-5169ccab87ed'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:32.757335 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e8a7f732-5fb7-4dc5-b7ef-1256ad385516 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1af83dff7a9e4e6d9e8bc15223a54096'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:32.787607 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e8a7f732-5fb7-4dc5-b7ef-1256ad385516 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1af83dff7a9e4e6d9e8bc15223a54096'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:32.798471 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e7a5ed2e-662b-42ba-8227-5169ccab87ed admin admin] The token cache is being invalidate because project 4fec6aa2d0c54645a84c040a7ae46ce4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:18:32.799221 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e7a5ed2e-662b-42ba-8227-5169ccab87ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:32.802332 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e7a5ed2e-662b-42ba-8227-5169ccab87ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:32.804469 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1090/2166] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:18:32 2022] DELETE /identity/v3/projects/4fec6aa2d0c54645a84c040a7ae46ce4 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:18:32.816673 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1077/2167] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:18:32 2022] DELETE /identity/v3/users/1af83dff7a9e4e6d9e8bc15223a54096 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:18:32.818952 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42db4119-c771-4884-8d0f-f75b0a7c786c None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:32.819257 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42db4119-c771-4884-8d0f-f75b0a7c786c None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:32.819532 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42db4119-c771-4884-8d0f-f75b0a7c786c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:32.875695 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-42db4119-c771-4884-8d0f-f75b0a7c786c None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:18:32.884349 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-42db4119-c771-4884-8d0f-f75b0a7c786c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:33.027897 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a7f732-5fb7-4dc5-b7ef-1256ad385516 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:33.030471 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e8a7f732-5fb7-4dc5-b7ef-1256ad385516 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:33.077056 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1091/2168] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:18:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:18:33.099282 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42db4119-c771-4884-8d0f-f75b0a7c786c None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:33.101495 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-42db4119-c771-4884-8d0f-f75b0a7c786c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:33.145412 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a7f732-5fb7-4dc5-b7ef-1256ad385516 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:33.151142 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1ce848-d861-47b8-af97-3e802b29df48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:33.152083 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ce848-d861-47b8-af97-3e802b29df48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:33.152083 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ce848-d861-47b8-af97-3e802b29df48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:33.152369 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ce848-d861-47b8-af97-3e802b29df48 admin admin] PATH_INFO: `/v3/users/a320a7e5bd7e4fd792d196a2745c8a1f` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:33.154276 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1ce848-d861-47b8-af97-3e802b29df48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a320a7e5bd7e4fd792d196a2745c8a1f)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:33.156315 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1ce848-d861-47b8-af97-3e802b29df48 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:33.207714 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:18:33.208931 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42db4119-c771-4884-8d0f-f75b0a7c786c None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:33.226585 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99b4e92-df68-43bc-baad-e7012bf69df3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:33.227850 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99b4e92-df68-43bc-baad-e7012bf69df3 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:33.228370 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99b4e92-df68-43bc-baad-e7012bf69df3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:33.229001 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99b4e92-df68-43bc-baad-e7012bf69df3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:33.263393 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99b4e92-df68-43bc-baad-e7012bf69df3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:33.271621 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99b4e92-df68-43bc-baad-e7012bf69df3 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:33.293047 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3d1ce848-d861-47b8-af97-3e802b29df48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a320a7e5bd7e4fd792d196a2745c8a1f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:33.295279 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3d1ce848-d861-47b8-af97-3e802b29df48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a320a7e5bd7e4fd792d196a2745c8a1f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:33.301638 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1092/2169] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:18:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:18:33.312967 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3d1ce848-d861-47b8-af97-3e802b29df48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a320a7e5bd7e4fd792d196a2745c8a1f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:33.345524 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3d1ce848-d861-47b8-af97-3e802b29df48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a320a7e5bd7e4fd792d196a2745c8a1f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:33.364474 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3d1ce848-d861-47b8-af97-3e802b29df48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a320a7e5bd7e4fd792d196a2745c8a1f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:33.374231 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1078/2170] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:18:33 2022] DELETE /identity/v3/users/a320a7e5bd7e4fd792d196a2745c8a1f => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:18:33.378760 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfcd4db-41ef-4003-a23d-fb058469269d None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:33.379055 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfcd4db-41ef-4003-a23d-fb058469269d None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:33.379312 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfcd4db-41ef-4003-a23d-fb058469269d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:33.408802 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-bcfcd4db-41ef-4003-a23d-fb058469269d None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:18:33.408802 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-bcfcd4db-41ef-4003-a23d-fb058469269d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:33.476410 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:18:33.500193 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1093/2171] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:18:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:18:33.513106 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1ce848-d861-47b8-af97-3e802b29df48 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:33.514667 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-3d1ce848-d861-47b8-af97-3e802b29df48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:33.583204 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1ce848-d861-47b8-af97-3e802b29df48 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:33.589015 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce9ed5c-ebef-4b18-8118-255499e03b98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:33.589751 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce9ed5c-ebef-4b18-8118-255499e03b98 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:33.590094 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce9ed5c-ebef-4b18-8118-255499e03b98 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:33.590369 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce9ed5c-ebef-4b18-8118-255499e03b98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:33.604805 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce9ed5c-ebef-4b18-8118-255499e03b98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:33.606821 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce9ed5c-ebef-4b18-8118-255499e03b98 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:33.620357 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1079/2172] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:18:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:18:33.825026 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfcd4db-41ef-4003-a23d-fb058469269d None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:33.826855 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-bcfcd4db-41ef-4003-a23d-fb058469269d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:33.847291 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfcd4db-41ef-4003-a23d-fb058469269d None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:33.854129 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0afb8da-3653-4047-81fa-28124206a55c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:33.854790 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0afb8da-3653-4047-81fa-28124206a55c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:33.855240 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0afb8da-3653-4047-81fa-28124206a55c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:33.855661 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0afb8da-3653-4047-81fa-28124206a55c admin admin] PATH_INFO: `/v3/projects/7b7d42cddc04425c99641db191f56de6` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:33.857414 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0afb8da-3653-4047-81fa-28124206a55c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b7d42cddc04425c99641db191f56de6)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:33.860007 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0afb8da-3653-4047-81fa-28124206a55c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:33.964384 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b0afb8da-3653-4047-81fa-28124206a55c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b7d42cddc04425c99641db191f56de6', 'request_id': 'req-b0afb8da-3653-4047-81fa-28124206a55c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:33.978932 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b0afb8da-3653-4047-81fa-28124206a55c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b7d42cddc04425c99641db191f56de6', 'request_id': 'req-b0afb8da-3653-4047-81fa-28124206a55c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:33.998119 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b0afb8da-3653-4047-81fa-28124206a55c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b7d42cddc04425c99641db191f56de6', 'request_id': 'req-b0afb8da-3653-4047-81fa-28124206a55c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:34.027736 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b0afb8da-3653-4047-81fa-28124206a55c admin admin] The token cache is being invalidate because project 7b7d42cddc04425c99641db191f56de6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:18:34.028028 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b0afb8da-3653-4047-81fa-28124206a55c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:34.028733 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b0afb8da-3653-4047-81fa-28124206a55c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:34.030618 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1094/2173] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:18:33 2022] DELETE /identity/v3/projects/7b7d42cddc04425c99641db191f56de6 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:18:34.034769 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce9ed5c-ebef-4b18-8118-255499e03b98 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:34.046588 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce9ed5c-ebef-4b18-8118-255499e03b98 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:34.105452 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0afb8da-3653-4047-81fa-28124206a55c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:34.108801 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b0afb8da-3653-4047-81fa-28124206a55c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:34.116210 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a703c1f-42c0-412d-9c90-bec8206db3db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:34.116913 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a703c1f-42c0-412d-9c90-bec8206db3db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:34.117193 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a703c1f-42c0-412d-9c90-bec8206db3db admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:34.117459 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a703c1f-42c0-412d-9c90-bec8206db3db admin admin] PATH_INFO: `/v3/users/c2d5b77c08774470816e3ddf2bb3be2c` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:34.118673 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a703c1f-42c0-412d-9c90-bec8206db3db admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2d5b77c08774470816e3ddf2bb3be2c)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:34.121993 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a703c1f-42c0-412d-9c90-bec8206db3db admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:34.151354 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0afb8da-3653-4047-81fa-28124206a55c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:34.183284 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8877b5-b3c2-4ed5-901c-9e19c218188f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:34.184298 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8877b5-b3c2-4ed5-901c-9e19c218188f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:34.184872 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8877b5-b3c2-4ed5-901c-9e19c218188f admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:34.185298 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8877b5-b3c2-4ed5-901c-9e19c218188f admin admin] PATH_INFO: `/v3/projects/12154ec79bc84b0b899de1bc94db3982` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:34.185416 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:18:34.188108 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-0a703c1f-42c0-412d-9c90-bec8206db3db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2d5b77c08774470816e3ddf2bb3be2c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:34.189024 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-0a703c1f-42c0-412d-9c90-bec8206db3db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2d5b77c08774470816e3ddf2bb3be2c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:34.191359 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8877b5-b3c2-4ed5-901c-9e19c218188f admin admin] RBAC: Authorizing `identity:delete_project(project_id=12154ec79bc84b0b899de1bc94db3982)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:34.194073 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8877b5-b3c2-4ed5-901c-9e19c218188f admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:34.200130 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-0a703c1f-42c0-412d-9c90-bec8206db3db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2d5b77c08774470816e3ddf2bb3be2c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:34.218507 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-0a703c1f-42c0-412d-9c90-bec8206db3db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2d5b77c08774470816e3ddf2bb3be2c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:34.232702 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-0a703c1f-42c0-412d-9c90-bec8206db3db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2d5b77c08774470816e3ddf2bb3be2c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:34.246038 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1080/2174] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:18:34 2022] DELETE /identity/v3/users/c2d5b77c08774470816e3ddf2bb3be2c => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:18:34.305959 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7e8877b5-b3c2-4ed5-901c-9e19c218188f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12154ec79bc84b0b899de1bc94db3982', 'request_id': 'req-7e8877b5-b3c2-4ed5-901c-9e19c218188f'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:34.315871 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7e8877b5-b3c2-4ed5-901c-9e19c218188f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12154ec79bc84b0b899de1bc94db3982', 'request_id': 'req-7e8877b5-b3c2-4ed5-901c-9e19c218188f'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:34.358318 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7e8877b5-b3c2-4ed5-901c-9e19c218188f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12154ec79bc84b0b899de1bc94db3982', 'request_id': 'req-7e8877b5-b3c2-4ed5-901c-9e19c218188f'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:34.382168 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7e8877b5-b3c2-4ed5-901c-9e19c218188f admin admin] The token cache is being invalidate because project 12154ec79bc84b0b899de1bc94db3982 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:18:34.382536 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7e8877b5-b3c2-4ed5-901c-9e19c218188f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:34.384165 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7e8877b5-b3c2-4ed5-901c-9e19c218188f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:34.388843 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1095/2175] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:18:34 2022] DELETE /identity/v3/projects/12154ec79bc84b0b899de1bc94db3982 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:18:34.392927 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a703c1f-42c0-412d-9c90-bec8206db3db admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:34.395046 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0a703c1f-42c0-412d-9c90-bec8206db3db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:34.577852 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a703c1f-42c0-412d-9c90-bec8206db3db admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:34.705178 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4a8531-1482-417b-a66a-609d0babf4d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:34.707020 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a8531-1482-417b-a66a-609d0babf4d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:34.707467 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a8531-1482-417b-a66a-609d0babf4d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:34.707892 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a8531-1482-417b-a66a-609d0babf4d0 admin admin] PATH_INFO: `/v3/users/bab8bcc84152488e937c3dc603842bac` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:34.710568 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4a8531-1482-417b-a66a-609d0babf4d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bab8bcc84152488e937c3dc603842bac)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:34.713480 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4a8531-1482-417b-a66a-609d0babf4d0 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:34.782714 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-7a4a8531-1482-417b-a66a-609d0babf4d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bab8bcc84152488e937c3dc603842bac'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:34.784279 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-7a4a8531-1482-417b-a66a-609d0babf4d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bab8bcc84152488e937c3dc603842bac'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:34.797254 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-7a4a8531-1482-417b-a66a-609d0babf4d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bab8bcc84152488e937c3dc603842bac'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:34.818434 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:18:34.820212 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-7a4a8531-1482-417b-a66a-609d0babf4d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bab8bcc84152488e937c3dc603842bac'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:34.836518 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-7a4a8531-1482-417b-a66a-609d0babf4d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bab8bcc84152488e937c3dc603842bac'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:34.847468 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1081/2176] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:18:34 2022] DELETE /identity/v3/users/bab8bcc84152488e937c3dc603842bac => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:18:34.938628 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8877b5-b3c2-4ed5-901c-9e19c218188f admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:34.940890 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-7e8877b5-b3c2-4ed5-901c-9e19c218188f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:34.961712 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8877b5-b3c2-4ed5-901c-9e19c218188f admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:35.067559 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:18:35.082879 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a224adee-4958-4de7-944f-b9d4bf2c27ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:35.083836 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a224adee-4958-4de7-944f-b9d4bf2c27ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:35.084232 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a224adee-4958-4de7-944f-b9d4bf2c27ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:35.084683 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a224adee-4958-4de7-944f-b9d4bf2c27ee admin admin] PATH_INFO: `/v3/projects/b93250123e1344b78a9c9898a3dd7b4b` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:35.086072 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a224adee-4958-4de7-944f-b9d4bf2c27ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=b93250123e1344b78a9c9898a3dd7b4b)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:35.088075 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a224adee-4958-4de7-944f-b9d4bf2c27ee admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:35.159368 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a224adee-4958-4de7-944f-b9d4bf2c27ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b93250123e1344b78a9c9898a3dd7b4b', 'request_id': 'req-a224adee-4958-4de7-944f-b9d4bf2c27ee'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:35.167493 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a224adee-4958-4de7-944f-b9d4bf2c27ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b93250123e1344b78a9c9898a3dd7b4b', 'request_id': 'req-a224adee-4958-4de7-944f-b9d4bf2c27ee'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:35.177771 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a224adee-4958-4de7-944f-b9d4bf2c27ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b93250123e1344b78a9c9898a3dd7b4b', 'request_id': 'req-a224adee-4958-4de7-944f-b9d4bf2c27ee'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:35.193339 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a224adee-4958-4de7-944f-b9d4bf2c27ee admin admin] The token cache is being invalidate because project b93250123e1344b78a9c9898a3dd7b4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:18:35.193818 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a224adee-4958-4de7-944f-b9d4bf2c27ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:35.194847 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a224adee-4958-4de7-944f-b9d4bf2c27ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:35.196949 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1096/2177] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:18:34 2022] DELETE /identity/v3/projects/b93250123e1344b78a9c9898a3dd7b4b => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:18:35.202674 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4a8531-1482-417b-a66a-609d0babf4d0 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:35.205482 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7a4a8531-1482-417b-a66a-609d0babf4d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:35.223334 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4a8531-1482-417b-a66a-609d0babf4d0 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:35.304874 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9dab40-5f10-4ba5-8893-ec016fb6b4ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:35.306499 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9dab40-5f10-4ba5-8893-ec016fb6b4ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:35.306958 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9dab40-5f10-4ba5-8893-ec016fb6b4ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:35.307471 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9dab40-5f10-4ba5-8893-ec016fb6b4ac admin admin] PATH_INFO: `/v3/users/2c14dd1fe11641449b4c227f6c9c171f` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:35.309176 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9dab40-5f10-4ba5-8893-ec016fb6b4ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c14dd1fe11641449b4c227f6c9c171f)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:35.312839 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9dab40-5f10-4ba5-8893-ec016fb6b4ac admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:35.350237 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a224adee-4958-4de7-944f-b9d4bf2c27ee admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:35.354762 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a224adee-4958-4de7-944f-b9d4bf2c27ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:35.371774 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a224adee-4958-4de7-944f-b9d4bf2c27ee admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:35.384934 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0739c80-fd75-45d1-989a-5601fb9218b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:35.387017 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0739c80-fd75-45d1-989a-5601fb9218b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:35.388185 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0739c80-fd75-45d1-989a-5601fb9218b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:35.389319 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0739c80-fd75-45d1-989a-5601fb9218b2 admin admin] PATH_INFO: `/v3/projects/0812168eb3be4dfeaf87a0ac9a2afd0c` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:35.391222 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0739c80-fd75-45d1-989a-5601fb9218b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0812168eb3be4dfeaf87a0ac9a2afd0c)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:35.394043 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0739c80-fd75-45d1-989a-5601fb9218b2 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:35.397221 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8d9dab40-5f10-4ba5-8893-ec016fb6b4ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c14dd1fe11641449b4c227f6c9c171f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:35.398451 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8d9dab40-5f10-4ba5-8893-ec016fb6b4ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c14dd1fe11641449b4c227f6c9c171f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:35.412507 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8d9dab40-5f10-4ba5-8893-ec016fb6b4ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c14dd1fe11641449b4c227f6c9c171f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:35.428493 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8d9dab40-5f10-4ba5-8893-ec016fb6b4ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c14dd1fe11641449b4c227f6c9c171f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:35.439380 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8d9dab40-5f10-4ba5-8893-ec016fb6b4ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c14dd1fe11641449b4c227f6c9c171f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:35.448312 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1082/2178] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:18:35 2022] DELETE /identity/v3/users/2c14dd1fe11641449b4c227f6c9c171f => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:18:35.520481 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a0739c80-fd75-45d1-989a-5601fb9218b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0812168eb3be4dfeaf87a0ac9a2afd0c', 'request_id': 'req-a0739c80-fd75-45d1-989a-5601fb9218b2'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:35.527417 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a0739c80-fd75-45d1-989a-5601fb9218b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0812168eb3be4dfeaf87a0ac9a2afd0c', 'request_id': 'req-a0739c80-fd75-45d1-989a-5601fb9218b2'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:35.545932 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a0739c80-fd75-45d1-989a-5601fb9218b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0812168eb3be4dfeaf87a0ac9a2afd0c', 'request_id': 'req-a0739c80-fd75-45d1-989a-5601fb9218b2'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:35.582206 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a0739c80-fd75-45d1-989a-5601fb9218b2 admin admin] The token cache is being invalidate because project 0812168eb3be4dfeaf87a0ac9a2afd0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:18:35.582206 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a0739c80-fd75-45d1-989a-5601fb9218b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:35.582206 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a0739c80-fd75-45d1-989a-5601fb9218b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:35.585104 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1097/2179] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:18:35 2022] DELETE /identity/v3/projects/0812168eb3be4dfeaf87a0ac9a2afd0c => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:18:35.590119 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da012ad-cf8a-4ac8-b10d-962a228c492d None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:35.590670 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da012ad-cf8a-4ac8-b10d-962a228c492d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:35.590670 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da012ad-cf8a-4ac8-b10d-962a228c492d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:35.615172 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-4da012ad-cf8a-4ac8-b10d-962a228c492d None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:18:35.616951 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-4da012ad-cf8a-4ac8-b10d-962a228c492d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:35.700115 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1083/2180] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:18:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:18:35.709389 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b9f3c6-99e7-441d-8a37-ee1526193bee None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:35.709752 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b9f3c6-99e7-441d-8a37-ee1526193bee None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:35.710091 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b9f3c6-99e7-441d-8a37-ee1526193bee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:35.751200 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-c8b9f3c6-99e7-441d-8a37-ee1526193bee None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:18:35.752812 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-c8b9f3c6-99e7-441d-8a37-ee1526193bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:35.784225 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1098/2181] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:18:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:18:35.788551 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da012ad-cf8a-4ac8-b10d-962a228c492d None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:35.803898 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da012ad-cf8a-4ac8-b10d-962a228c492d None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:35.820854 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5131207-7f70-4ae0-b981-79c2c01db3ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:35.821744 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5131207-7f70-4ae0-b981-79c2c01db3ba admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:35.822108 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5131207-7f70-4ae0-b981-79c2c01db3ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:35.822488 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5131207-7f70-4ae0-b981-79c2c01db3ba admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:35.823264 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5131207-7f70-4ae0-b981-79c2c01db3ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:18:35.823636 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5131207-7f70-4ae0-b981-79c2c01db3ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:35.827079 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5131207-7f70-4ae0-b981-79c2c01db3ba admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:35.836417 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:18:35.853574 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1084/2182] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 6 16:18:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:18:35.866817 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b9f3c6-99e7-441d-8a37-ee1526193bee None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:35.879899 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b9f3c6-99e7-441d-8a37-ee1526193bee None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:35.886883 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619f1383-89b8-45d4-a419-78ef12c2e651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:35.888314 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f1383-89b8-45d4-a419-78ef12c2e651 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:35.888874 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f1383-89b8-45d4-a419-78ef12c2e651 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:35.889374 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f1383-89b8-45d4-a419-78ef12c2e651 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:35.890240 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619f1383-89b8-45d4-a419-78ef12c2e651 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:35.892966 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619f1383-89b8-45d4-a419-78ef12c2e651 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:35.949082 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1099/2183] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:18:35 2022] POST /identity/v3/projects => generated 369 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:18:35.953437 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5131207-7f70-4ae0-b981-79c2c01db3ba admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:35.968871 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5131207-7f70-4ae0-b981-79c2c01db3ba admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:36.149811 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc44b406-5068-4bd9-8716-ee3fcc4206ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:36.150472 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc44b406-5068-4bd9-8716-ee3fcc4206ca admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:36.150750 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc44b406-5068-4bd9-8716-ee3fcc4206ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:36.151070 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc44b406-5068-4bd9-8716-ee3fcc4206ca admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:36.151670 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc44b406-5068-4bd9-8716-ee3fcc4206ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:36.154747 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc44b406-5068-4bd9-8716-ee3fcc4206ca admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:36.198489 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1085/2184] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:18:35 2022] POST /identity/v3/users => generated 285 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:18:36.204241 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619f1383-89b8-45d4-a419-78ef12c2e651 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:36.233728 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619f1383-89b8-45d4-a419-78ef12c2e651 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:36.245549 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2282a373-c4f6-4f7f-a15e-bfcb17a0eac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:36.254348 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2282a373-c4f6-4f7f-a15e-bfcb17a0eac7 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:36.255296 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2282a373-c4f6-4f7f-a15e-bfcb17a0eac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:36.255960 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2282a373-c4f6-4f7f-a15e-bfcb17a0eac7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:36.257762 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2282a373-c4f6-4f7f-a15e-bfcb17a0eac7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:36.261322 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2282a373-c4f6-4f7f-a15e-bfcb17a0eac7 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:36.278055 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1100/2185] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:18:36 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 16:18:36.284199 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc44b406-5068-4bd9-8716-ee3fcc4206ca admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:36.303237 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2282a373-c4f6-4f7f-a15e-bfcb17a0eac7 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:36.305733 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc44b406-5068-4bd9-8716-ee3fcc4206ca admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:36.307288 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-2282a373-c4f6-4f7f-a15e-bfcb17a0eac7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:36.327839 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2282a373-c4f6-4f7f-a15e-bfcb17a0eac7 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:36.334969 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f3445b-1836-49d0-baec-230739ea68ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:36.336218 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f3445b-1836-49d0-baec-230739ea68ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:36.336780 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f3445b-1836-49d0-baec-230739ea68ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:36.337310 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f3445b-1836-49d0-baec-230739ea68ea admin admin] PATH_INFO: `/v3/projects/5e48edaea09c469c96ae6baadce14a9b` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:36.339299 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f3445b-1836-49d0-baec-230739ea68ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e48edaea09c469c96ae6baadce14a9b)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:36.342186 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f3445b-1836-49d0-baec-230739ea68ea admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:36.359617 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c029df-a071-47a3-9c09-f6b7c5de87b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:36.362881 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c029df-a071-47a3-9c09-f6b7c5de87b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:36.363176 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c029df-a071-47a3-9c09-f6b7c5de87b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:36.363555 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c029df-a071-47a3-9c09-f6b7c5de87b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:36.364230 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c029df-a071-47a3-9c09-f6b7c5de87b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:36.365804 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c029df-a071-47a3-9c09-f6b7c5de87b5 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:36.403406 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1086/2186] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:18:36 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:18:36.411434 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c029df-a071-47a3-9c09-f6b7c5de87b5 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:36.416269 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=82,keystone:DELETE=15,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:18:36.438494 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c029df-a071-47a3-9c09-f6b7c5de87b5 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:36.455975 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-37f3445b-1836-49d0-baec-230739ea68ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e48edaea09c469c96ae6baadce14a9b', 'request_id': 'req-37f3445b-1836-49d0-baec-230739ea68ea'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:36.467033 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-37f3445b-1836-49d0-baec-230739ea68ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e48edaea09c469c96ae6baadce14a9b', 'request_id': 'req-37f3445b-1836-49d0-baec-230739ea68ea'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:36.483694 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-37f3445b-1836-49d0-baec-230739ea68ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e48edaea09c469c96ae6baadce14a9b', 'request_id': 'req-37f3445b-1836-49d0-baec-230739ea68ea'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:36.501865 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-37f3445b-1836-49d0-baec-230739ea68ea admin admin] The token cache is being invalidate because project 5e48edaea09c469c96ae6baadce14a9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:18:36.502800 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-37f3445b-1836-49d0-baec-230739ea68ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:36.503398 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-37f3445b-1836-49d0-baec-230739ea68ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:36.505026 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1101/2187] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:18:36 2022] DELETE /identity/v3/projects/5e48edaea09c469c96ae6baadce14a9b => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:18:36.511127 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01adf356-0fcd-4423-8997-5d5654781ce7 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:36.511769 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01adf356-0fcd-4423-8997-5d5654781ce7 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:36.512203 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01adf356-0fcd-4423-8997-5d5654781ce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:36.526294 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df44360-a284-490f-b4f0-18587166115e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:36.527241 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df44360-a284-490f-b4f0-18587166115e admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:36.527586 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df44360-a284-490f-b4f0-18587166115e admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:36.527941 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df44360-a284-490f-b4f0-18587166115e admin admin] PATH_INFO: `/v3/projects/876fa8cb471c4f06bf12518e0439f384/users/7cdbaac37cff41feb5441af8eb1d1c73/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:36.537436 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-01adf356-0fcd-4423-8997-5d5654781ce7 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:18:36.538121 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-01adf356-0fcd-4423-8997-5d5654781ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:36.555325 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1102/2188] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:18:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:18:36.563989 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c906f4-1cf6-4a5d-8d81-28c5ff134132 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:36.564238 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c906f4-1cf6-4a5d-8d81-28c5ff134132 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:36.564511 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c906f4-1cf6-4a5d-8d81-28c5ff134132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:36.572243 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df44360-a284-490f-b4f0-18587166115e admin admin] RBAC: Authorizing `identity:create_grant(project_id=876fa8cb471c4f06bf12518e0439f384, user_id=7cdbaac37cff41feb5441af8eb1d1c73, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:36.577318 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df44360-a284-490f-b4f0-18587166115e admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:36.593652 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-b3c906f4-1cf6-4a5d-8d81-28c5ff134132 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:18:36.596816 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b3c906f4-1cf6-4a5d-8d81-28c5ff134132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:36.600703 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1087/2189] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:18:36 2022] PUT /identity/v3/projects/876fa8cb471c4f06bf12518e0439f384/users/7cdbaac37cff41feb5441af8eb1d1c73/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:18:36.613910 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd14557-1205-4164-b1b8-3b370863e104 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:36.614307 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd14557-1205-4164-b1b8-3b370863e104 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:36.614668 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd14557-1205-4164-b1b8-3b370863e104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:36.665436 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-5cd14557-1205-4164-b1b8-3b370863e104 None None] MFA Rules not processed for user `7cdbaac37cff41feb5441af8eb1d1c73`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:18:36.666952 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5cd14557-1205-4164-b1b8-3b370863e104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:36.708517 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1103/2190] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:18:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:18:36.712879 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c906f4-1cf6-4a5d-8d81-28c5ff134132 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:36.730393 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c906f4-1cf6-4a5d-8d81-28c5ff134132 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:36.736265 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:18:36.744959 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1933ea3d-8750-4e67-b4d1-ea4130240a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:36.746123 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1933ea3d-8750-4e67-b4d1-ea4130240a97 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:36.746665 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1933ea3d-8750-4e67-b4d1-ea4130240a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:36.747244 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1933ea3d-8750-4e67-b4d1-ea4130240a97 admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:36.748428 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1933ea3d-8750-4e67-b4d1-ea4130240a97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:18:36.749057 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1933ea3d-8750-4e67-b4d1-ea4130240a97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:36.752128 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1933ea3d-8750-4e67-b4d1-ea4130240a97 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:36.768618 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1088/2191] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:18:36 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:18:36.773608 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd14557-1205-4164-b1b8-3b370863e104 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:36.774725 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1104/2192] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 6 16:18:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:18:36.774980 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5cd14557-1205-4164-b1b8-3b370863e104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:36.778922 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1933ea3d-8750-4e67-b4d1-ea4130240a97 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:36.788629 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd14557-1205-4164-b1b8-3b370863e104 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:36.789002 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1933ea3d-8750-4e67-b4d1-ea4130240a97 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:36.794514 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310bd2e1-ba81-4d0c-8db3-942e59f5a078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:36.795236 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310bd2e1-ba81-4d0c-8db3-942e59f5a078 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:36.795525 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310bd2e1-ba81-4d0c-8db3-942e59f5a078 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:36.795636 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf16f3d1-7ce2-4552-b662-ee7c46aa4edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:36.795807 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310bd2e1-ba81-4d0c-8db3-942e59f5a078 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:36.796193 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf16f3d1-7ce2-4552-b662-ee7c46aa4edf admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:36.796446 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310bd2e1-ba81-4d0c-8db3-942e59f5a078 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:36.796567 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf16f3d1-7ce2-4552-b662-ee7c46aa4edf admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:36.796740 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf16f3d1-7ce2-4552-b662-ee7c46aa4edf admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:36.797421 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf16f3d1-7ce2-4552-b662-ee7c46aa4edf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:36.798495 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310bd2e1-ba81-4d0c-8db3-942e59f5a078 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:36.799388 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf16f3d1-7ce2-4552-b662-ee7c46aa4edf admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:36.842061 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1089/2193] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:18:36 2022] POST /identity/v3/projects => generated 369 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:18:36.843541 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310bd2e1-ba81-4d0c-8db3-942e59f5a078 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:36.855834 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1105/2194] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:18:36 2022] POST /identity/v3/projects => generated 383 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:18:36.857147 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf16f3d1-7ce2-4552-b662-ee7c46aa4edf admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:36.938417 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf16f3d1-7ce2-4552-b662-ee7c46aa4edf admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:36.939808 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310bd2e1-ba81-4d0c-8db3-942e59f5a078 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:36.943733 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735689aa-e375-47e6-b385-60e8d882adde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:36.944447 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735689aa-e375-47e6-b385-60e8d882adde admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:36.944736 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735689aa-e375-47e6-b385-60e8d882adde admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:36.945047 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735689aa-e375-47e6-b385-60e8d882adde admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:36.945946 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735689aa-e375-47e6-b385-60e8d882adde admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:36.947433 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17662dd5-1456-472b-b3f8-5fbdf14ca8ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:36.948122 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735689aa-e375-47e6-b385-60e8d882adde admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:36.948448 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17662dd5-1456-472b-b3f8-5fbdf14ca8ef admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:36.948967 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17662dd5-1456-472b-b3f8-5fbdf14ca8ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:36.949437 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17662dd5-1456-472b-b3f8-5fbdf14ca8ef admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:36.950195 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17662dd5-1456-472b-b3f8-5fbdf14ca8ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:36.952199 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17662dd5-1456-472b-b3f8-5fbdf14ca8ef admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:36.976813 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1090/2195] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:18:36 2022] POST /identity/v3/users => generated 291 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:18:36.977355 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1106/2196] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:18:36 2022] POST /identity/v3/users => generated 292 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:18:36.980967 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735689aa-e375-47e6-b385-60e8d882adde admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:36.983952 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17662dd5-1456-472b-b3f8-5fbdf14ca8ef admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:36.986672 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:18:36.992862 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735689aa-e375-47e6-b385-60e8d882adde admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:36.995724 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17662dd5-1456-472b-b3f8-5fbdf14ca8ef admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:36.999669 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc8dc6b-a504-4376-9e63-4cef84ba0ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:37.000346 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc8dc6b-a504-4376-9e63-4cef84ba0ca5 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:37.000632 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc8dc6b-a504-4376-9e63-4cef84ba0ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:37.001175 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c90cd4-f2b4-495b-9d4a-def11627521b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:37.001316 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc8dc6b-a504-4376-9e63-4cef84ba0ca5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:37.001799 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c90cd4-f2b4-495b-9d4a-def11627521b admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:37.002065 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc8dc6b-a504-4376-9e63-4cef84ba0ca5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:37.002168 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c90cd4-f2b4-495b-9d4a-def11627521b admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:37.002339 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c90cd4-f2b4-495b-9d4a-def11627521b admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:37.003015 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c90cd4-f2b4-495b-9d4a-def11627521b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:37.004672 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c90cd4-f2b4-495b-9d4a-def11627521b admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:37.004860 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc8dc6b-a504-4376-9e63-4cef84ba0ca5 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:37.016946 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1091/2197] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:18:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 16:18:37.021269 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c90cd4-f2b4-495b-9d4a-def11627521b admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:37.026671 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1107/2198] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:18:36 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 16:18:37.027937 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc8dc6b-a504-4376-9e63-4cef84ba0ca5 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:37.033547 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c90cd4-f2b4-495b-9d4a-def11627521b admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:37.039137 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8c0dad-21c8-4a18-9695-adfa676cc533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:37.040103 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c0dad-21c8-4a18-9695-adfa676cc533 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:37.040424 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc8dc6b-a504-4376-9e63-4cef84ba0ca5 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:37.040546 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c0dad-21c8-4a18-9695-adfa676cc533 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:37.040964 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c0dad-21c8-4a18-9695-adfa676cc533 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:37.042398 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8c0dad-21c8-4a18-9695-adfa676cc533 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:37.044407 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8c0dad-21c8-4a18-9695-adfa676cc533 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:37.045625 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a1487f-6f5a-4b1f-a4e3-824deb44faea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:37.046760 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a1487f-6f5a-4b1f-a4e3-824deb44faea admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:37.047046 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a1487f-6f5a-4b1f-a4e3-824deb44faea admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:37.047578 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a1487f-6f5a-4b1f-a4e3-824deb44faea admin admin] PATH_INFO: `/v3/projects/f9d32e6dfaf3452e95b0c67d34f6574d/users/c2ff2240cbb14d2f85253849fff8e10b/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:37.059324 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1092/2199] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:18:37 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 16:18:37.065013 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8c0dad-21c8-4a18-9695-adfa676cc533 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:37.071223 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a1487f-6f5a-4b1f-a4e3-824deb44faea admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9d32e6dfaf3452e95b0c67d34f6574d, user_id=c2ff2240cbb14d2f85253849fff8e10b, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:37.077151 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8c0dad-21c8-4a18-9695-adfa676cc533 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:37.084164 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902ec6ef-8f44-4e6f-b59e-79a603296d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:37.084690 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a1487f-6f5a-4b1f-a4e3-824deb44faea admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:37.084885 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902ec6ef-8f44-4e6f-b59e-79a603296d16 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:37.085223 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902ec6ef-8f44-4e6f-b59e-79a603296d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:37.085506 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902ec6ef-8f44-4e6f-b59e-79a603296d16 admin admin] PATH_INFO: `/v3/projects/c6ce6188a2e24ce5ad940de5b8f57f44/users/bdbedac701d84205b607f6112ea3a109/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:37.109852 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902ec6ef-8f44-4e6f-b59e-79a603296d16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6ce6188a2e24ce5ad940de5b8f57f44, user_id=bdbedac701d84205b607f6112ea3a109, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:37.113898 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902ec6ef-8f44-4e6f-b59e-79a603296d16 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:37.136993 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1093/2200] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:18:37 2022] PUT /identity/v3/projects/c6ce6188a2e24ce5ad940de5b8f57f44/users/bdbedac701d84205b607f6112ea3a109/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:18:37.150439 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53bd61a-cb6e-4a78-930e-4d2db5eb402e None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:37.151525 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53bd61a-cb6e-4a78-930e-4d2db5eb402e None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:37.151916 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53bd61a-cb6e-4a78-930e-4d2db5eb402e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:37.161472 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1108/2201] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:18:37 2022] PUT /identity/v3/projects/f9d32e6dfaf3452e95b0c67d34f6574d/users/c2ff2240cbb14d2f85253849fff8e10b/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:18:37.165633 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a1487f-6f5a-4b1f-a4e3-824deb44faea admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:37.218715 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-d53bd61a-cb6e-4a78-930e-4d2db5eb402e None None] MFA Rules not processed for user `bdbedac701d84205b607f6112ea3a109`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:18:37.221608 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d53bd61a-cb6e-4a78-930e-4d2db5eb402e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:37.337709 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a1487f-6f5a-4b1f-a4e3-824deb44faea admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:37.428405 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:18:37.441377 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1094/2202] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:18:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:18:37.453385 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53bd61a-cb6e-4a78-930e-4d2db5eb402e None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:37.455288 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d53bd61a-cb6e-4a78-930e-4d2db5eb402e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:37.475453 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add4a74f-8966-40b8-b4e3-42bd5fbd5286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:37.476737 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add4a74f-8966-40b8-b4e3-42bd5fbd5286 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:37.477987 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add4a74f-8966-40b8-b4e3-42bd5fbd5286 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:37.477987 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add4a74f-8966-40b8-b4e3-42bd5fbd5286 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:37.478851 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add4a74f-8966-40b8-b4e3-42bd5fbd5286 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:37.482672 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add4a74f-8966-40b8-b4e3-42bd5fbd5286 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:37.509394 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1109/2203] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:18:37 2022] GET /identity/v3/roles => generated 1420 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 16:18:37.516056 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add4a74f-8966-40b8-b4e3-42bd5fbd5286 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:37.544826 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add4a74f-8966-40b8-b4e3-42bd5fbd5286 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:37.559951 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3b8feb-01c3-402a-b10e-9bee45067d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:37.561606 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3b8feb-01c3-402a-b10e-9bee45067d70 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:37.562441 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3b8feb-01c3-402a-b10e-9bee45067d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:37.564144 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3b8feb-01c3-402a-b10e-9bee45067d70 admin admin] PATH_INFO: `/v3/projects/f9d32e6dfaf3452e95b0c67d34f6574d/users/c2ff2240cbb14d2f85253849fff8e10b/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:37.568353 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3b8feb-01c3-402a-b10e-9bee45067d70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9d32e6dfaf3452e95b0c67d34f6574d, user_id=c2ff2240cbb14d2f85253849fff8e10b, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:37.575666 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3b8feb-01c3-402a-b10e-9bee45067d70 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:37.593891 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1110/2204] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:18:37 2022] PUT /identity/v3/projects/f9d32e6dfaf3452e95b0c67d34f6574d/users/c2ff2240cbb14d2f85253849fff8e10b/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:18:37.596087 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53bd61a-cb6e-4a78-930e-4d2db5eb402e None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:37.604591 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e2038-a44f-4478-a66a-c340462accf9 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:37.605679 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e2038-a44f-4478-a66a-c340462accf9 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:37.605679 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e2038-a44f-4478-a66a-c340462accf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:37.652934 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-0c4e2038-a44f-4478-a66a-c340462accf9 None None] MFA Rules not processed for user `c2ff2240cbb14d2f85253849fff8e10b`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:18:37.654898 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0c4e2038-a44f-4478-a66a-c340462accf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:37.676604 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84679c6-ef37-420c-b9c2-5f99a41dcb43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:37.677273 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84679c6-ef37-420c-b9c2-5f99a41dcb43 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:37.677562 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84679c6-ef37-420c-b9c2-5f99a41dcb43 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:37.677838 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84679c6-ef37-420c-b9c2-5f99a41dcb43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:37.743446 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:18:37.770729 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84679c6-ef37-420c-b9c2-5f99a41dcb43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:37.772203 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84679c6-ef37-420c-b9c2-5f99a41dcb43 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:37.778772 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1111/2205] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:18:37 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:18:37.783837 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1095/2206] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:18:37 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:18:37.792309 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4e2038-a44f-4478-a66a-c340462accf9 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:37.794001 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0c4e2038-a44f-4478-a66a-c340462accf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:37.806945 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4e2038-a44f-4478-a66a-c340462accf9 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:37.813374 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab022ce7-8834-424b-b897-464125cf5323 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:37.814568 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab022ce7-8834-424b-b897-464125cf5323 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:37.815061 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab022ce7-8834-424b-b897-464125cf5323 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:37.815538 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab022ce7-8834-424b-b897-464125cf5323 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:37.850154 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab022ce7-8834-424b-b897-464125cf5323 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:37.851782 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab022ce7-8834-424b-b897-464125cf5323 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:37.897217 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1112/2207] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:18:37 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:18:37.961498 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84679c6-ef37-420c-b9c2-5f99a41dcb43 service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:37.971995 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84679c6-ef37-420c-b9c2-5f99a41dcb43 service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:37.976112 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc66a2a2-d058-4f22-821e-b93a0458565e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:37.977006 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc66a2a2-d058-4f22-821e-b93a0458565e service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:37.977283 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc66a2a2-d058-4f22-821e-b93a0458565e service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:37.977545 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc66a2a2-d058-4f22-821e-b93a0458565e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:38.069544 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc66a2a2-d058-4f22-821e-b93a0458565e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:38.071147 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc66a2a2-d058-4f22-821e-b93a0458565e service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:38.086376 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1096/2208] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:18:37 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:18:40.501799 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab022ce7-8834-424b-b897-464125cf5323 service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:40.567334 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab022ce7-8834-424b-b897-464125cf5323 service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:40.572136 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31e3c27-0ebe-4bb9-af7d-0f6a4c590878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:40.572772 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31e3c27-0ebe-4bb9-af7d-0f6a4c590878 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:40.573079 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31e3c27-0ebe-4bb9-af7d-0f6a4c590878 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:40.573357 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31e3c27-0ebe-4bb9-af7d-0f6a4c590878 admin admin] PATH_INFO: `/v3/users/bdbedac701d84205b607f6112ea3a109` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:40.574499 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31e3c27-0ebe-4bb9-af7d-0f6a4c590878 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdbedac701d84205b607f6112ea3a109)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:40.576439 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31e3c27-0ebe-4bb9-af7d-0f6a4c590878 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:40.641573 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a31e3c27-0ebe-4bb9-af7d-0f6a4c590878 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdbedac701d84205b607f6112ea3a109'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:40.642272 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a31e3c27-0ebe-4bb9-af7d-0f6a4c590878 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdbedac701d84205b607f6112ea3a109'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:40.649804 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a31e3c27-0ebe-4bb9-af7d-0f6a4c590878 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdbedac701d84205b607f6112ea3a109'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:40.663001 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a31e3c27-0ebe-4bb9-af7d-0f6a4c590878 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdbedac701d84205b607f6112ea3a109'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:40.663825 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a31e3c27-0ebe-4bb9-af7d-0f6a4c590878 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdbedac701d84205b607f6112ea3a109'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:40.678317 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1113/2209] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:18:40 2022] DELETE /identity/v3/users/bdbedac701d84205b607f6112ea3a109 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:18:40.683041 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02877334-27a4-4db0-add0-3e1c0c51d197 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:40.683529 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02877334-27a4-4db0-add0-3e1c0c51d197 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:40.683649 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02877334-27a4-4db0-add0-3e1c0c51d197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:40.697086 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:18:40.703232 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-02877334-27a4-4db0-add0-3e1c0c51d197 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:18:40.704481 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-02877334-27a4-4db0-add0-3e1c0c51d197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:40.794372 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1097/2210] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:18:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:18:40.805093 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31e3c27-0ebe-4bb9-af7d-0f6a4c590878 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:40.806658 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a31e3c27-0ebe-4bb9-af7d-0f6a4c590878 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:40.881765 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:18:40.886361 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31e3c27-0ebe-4bb9-af7d-0f6a4c590878 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:40.892962 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9880aa1b-841a-459f-896e-3c4a5624db5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:40.893835 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9880aa1b-841a-459f-896e-3c4a5624db5b service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:40.894135 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9880aa1b-841a-459f-896e-3c4a5624db5b service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:40.894634 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9880aa1b-841a-459f-896e-3c4a5624db5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:40.904589 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9880aa1b-841a-459f-896e-3c4a5624db5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:40.906077 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9880aa1b-841a-459f-896e-3c4a5624db5b service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:40.916246 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1114/2211] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:18:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:18:41.384470 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02877334-27a4-4db0-add0-3e1c0c51d197 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:41.384470 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-02877334-27a4-4db0-add0-3e1c0c51d197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:41.403756 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02877334-27a4-4db0-add0-3e1c0c51d197 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:41.413901 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f08f70-570d-441c-ad73-678f5c777d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:41.414121 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f08f70-570d-441c-ad73-678f5c777d67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:41.414218 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f08f70-570d-441c-ad73-678f5c777d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:41.414481 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f08f70-570d-441c-ad73-678f5c777d67 admin admin] PATH_INFO: `/v3/projects/c6ce6188a2e24ce5ad940de5b8f57f44` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:41.415721 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f08f70-570d-441c-ad73-678f5c777d67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6ce6188a2e24ce5ad940de5b8f57f44)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:41.417745 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f08f70-570d-441c-ad73-678f5c777d67 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:41.524721 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c3f08f70-570d-441c-ad73-678f5c777d67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6ce6188a2e24ce5ad940de5b8f57f44', 'request_id': 'req-c3f08f70-570d-441c-ad73-678f5c777d67'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:41.533187 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c3f08f70-570d-441c-ad73-678f5c777d67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6ce6188a2e24ce5ad940de5b8f57f44', 'request_id': 'req-c3f08f70-570d-441c-ad73-678f5c777d67'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:41.540439 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c3f08f70-570d-441c-ad73-678f5c777d67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6ce6188a2e24ce5ad940de5b8f57f44', 'request_id': 'req-c3f08f70-570d-441c-ad73-678f5c777d67'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:41.553795 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c3f08f70-570d-441c-ad73-678f5c777d67 admin admin] The token cache is being invalidate because project c6ce6188a2e24ce5ad940de5b8f57f44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:18:41.554082 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c3f08f70-570d-441c-ad73-678f5c777d67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:41.554756 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c3f08f70-570d-441c-ad73-678f5c777d67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:41.556247 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1098/2212] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:18:41 2022] DELETE /identity/v3/projects/c6ce6188a2e24ce5ad940de5b8f57f44 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:18:41.561572 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a374e55-8f87-46d7-b3b5-937c092ef52a None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:41.562072 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a374e55-8f87-46d7-b3b5-937c092ef52a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:41.562196 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a374e55-8f87-46d7-b3b5-937c092ef52a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:41.582542 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-4a374e55-8f87-46d7-b3b5-937c092ef52a None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:18:41.583962 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-4a374e55-8f87-46d7-b3b5-937c092ef52a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:41.712583 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1115/2213] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:18:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:18:41.718835 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4608489f-d21b-43e9-b9b0-97814479afdb None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:41.719610 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4608489f-d21b-43e9-b9b0-97814479afdb None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:41.719610 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4608489f-d21b-43e9-b9b0-97814479afdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:41.744289 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-4608489f-d21b-43e9-b9b0-97814479afdb None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:18:41.745630 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-4608489f-d21b-43e9-b9b0-97814479afdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:41.753750 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1099/2214] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:18:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:18:41.758220 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a374e55-8f87-46d7-b3b5-937c092ef52a None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:41.768436 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a374e55-8f87-46d7-b3b5-937c092ef52a None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:41.773344 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af254dc-546a-4660-99d8-a4154a1a1c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:41.774040 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af254dc-546a-4660-99d8-a4154a1a1c16 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:41.774399 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af254dc-546a-4660-99d8-a4154a1a1c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:41.774734 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af254dc-546a-4660-99d8-a4154a1a1c16 admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:41.775470 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af254dc-546a-4660-99d8-a4154a1a1c16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:18:41.775837 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af254dc-546a-4660-99d8-a4154a1a1c16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:41.777627 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af254dc-546a-4660-99d8-a4154a1a1c16 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:41.790583 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1116/2215] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 6 16:18:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:18:41.795232 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4608489f-d21b-43e9-b9b0-97814479afdb None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:41.805356 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4608489f-d21b-43e9-b9b0-97814479afdb None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:41.810613 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192fb4a0-4293-4d07-893a-22cd28e7dcc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:41.811322 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192fb4a0-4293-4d07-893a-22cd28e7dcc3 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:41.811661 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192fb4a0-4293-4d07-893a-22cd28e7dcc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:41.811990 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192fb4a0-4293-4d07-893a-22cd28e7dcc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:41.812704 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192fb4a0-4293-4d07-893a-22cd28e7dcc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:41.814742 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192fb4a0-4293-4d07-893a-22cd28e7dcc3 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:41.856013 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1100/2216] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:18:41 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:18:41.857458 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af254dc-546a-4660-99d8-a4154a1a1c16 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:41.873805 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af254dc-546a-4660-99d8-a4154a1a1c16 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:41.962529 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c603b85f-d2cb-4403-b42d-c7fe867766a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:41.963064 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c603b85f-d2cb-4403-b42d-c7fe867766a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:41.963711 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c603b85f-d2cb-4403-b42d-c7fe867766a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:41.964944 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c603b85f-d2cb-4403-b42d-c7fe867766a1 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:41.966250 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c603b85f-d2cb-4403-b42d-c7fe867766a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:41.967521 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c603b85f-d2cb-4403-b42d-c7fe867766a1 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:42.055105 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1117/2217] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:18:41 2022] POST /identity/v3/users => generated 287 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:18:42.058728 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192fb4a0-4293-4d07-893a-22cd28e7dcc3 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:42.084635 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192fb4a0-4293-4d07-893a-22cd28e7dcc3 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:42.089529 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16d724d-7f9f-4dd0-b3df-0fdf7c672884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:42.090150 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d724d-7f9f-4dd0-b3df-0fdf7c672884 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:42.090430 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d724d-7f9f-4dd0-b3df-0fdf7c672884 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:42.090698 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d724d-7f9f-4dd0-b3df-0fdf7c672884 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:42.091350 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16d724d-7f9f-4dd0-b3df-0fdf7c672884 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:42.093017 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16d724d-7f9f-4dd0-b3df-0fdf7c672884 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:42.104774 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1101/2218] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:18:42 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:18:42.111484 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c603b85f-d2cb-4403-b42d-c7fe867766a1 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:42.134428 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c603b85f-d2cb-4403-b42d-c7fe867766a1 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:42.139702 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a947ebb-9c2d-4a02-8d29-46134f5882c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:42.140321 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a947ebb-9c2d-4a02-8d29-46134f5882c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:42.140603 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a947ebb-9c2d-4a02-8d29-46134f5882c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:42.140898 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a947ebb-9c2d-4a02-8d29-46134f5882c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:42.141551 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a947ebb-9c2d-4a02-8d29-46134f5882c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:42.143249 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a947ebb-9c2d-4a02-8d29-46134f5882c7 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:42.164832 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1118/2219] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:18:42 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:18:42.167337 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16d724d-7f9f-4dd0-b3df-0fdf7c672884 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:42.177845 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16d724d-7f9f-4dd0-b3df-0fdf7c672884 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:42.182730 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd44148a-45e7-4044-887a-abd8fa02c04b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:42.183380 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd44148a-45e7-4044-887a-abd8fa02c04b admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:42.183654 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd44148a-45e7-4044-887a-abd8fa02c04b admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:42.183919 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd44148a-45e7-4044-887a-abd8fa02c04b admin admin] PATH_INFO: `/v3/projects/681a935ab92448daa87d7cab3c70d630/users/04fa1b7be92a47dbb023a3bb94519323/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:42.202627 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd44148a-45e7-4044-887a-abd8fa02c04b admin admin] RBAC: Authorizing `identity:create_grant(project_id=681a935ab92448daa87d7cab3c70d630, user_id=04fa1b7be92a47dbb023a3bb94519323, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:42.205402 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd44148a-45e7-4044-887a-abd8fa02c04b admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:42.221626 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1102/2220] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:18:42 2022] PUT /identity/v3/projects/681a935ab92448daa87d7cab3c70d630/users/04fa1b7be92a47dbb023a3bb94519323/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:18:42.232339 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc4146-47be-4f98-8ca5-8a4c6966ffe9 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:42.232971 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc4146-47be-4f98-8ca5-8a4c6966ffe9 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:42.233461 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc4146-47be-4f98-8ca5-8a4c6966ffe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:42.283578 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-9fbc4146-47be-4f98-8ca5-8a4c6966ffe9 None None] MFA Rules not processed for user `04fa1b7be92a47dbb023a3bb94519323`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:18:42.285334 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-9fbc4146-47be-4f98-8ca5-8a4c6966ffe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:42.315461 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd44148a-45e7-4044-887a-abd8fa02c04b admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:42.317221 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-dd44148a-45e7-4044-887a-abd8fa02c04b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:42.351072 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:18:42.387619 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:18:42.396252 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1119/2221] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:18:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:18:42.407775 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbc4146-47be-4f98-8ca5-8a4c6966ffe9 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:42.410026 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-9fbc4146-47be-4f98-8ca5-8a4c6966ffe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:42.439182 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd44148a-45e7-4044-887a-abd8fa02c04b admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:42.444949 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4d2bc7-0c1e-4533-9e61-6ca9293a5908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:42.445809 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4d2bc7-0c1e-4533-9e61-6ca9293a5908 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:42.446428 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4d2bc7-0c1e-4533-9e61-6ca9293a5908 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:42.446804 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4d2bc7-0c1e-4533-9e61-6ca9293a5908 admin admin] PATH_INFO: `/v3/users/7cdbaac37cff41feb5441af8eb1d1c73` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:42.448594 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4d2bc7-0c1e-4533-9e61-6ca9293a5908 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cdbaac37cff41feb5441af8eb1d1c73)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:42.452047 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4d2bc7-0c1e-4533-9e61-6ca9293a5908 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:42.505054 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbc4146-47be-4f98-8ca5-8a4c6966ffe9 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:42.509959 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad238ad-e77a-4c57-8161-f4c071215fa5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:42.510677 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad238ad-e77a-4c57-8161-f4c071215fa5 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:42.511161 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad238ad-e77a-4c57-8161-f4c071215fa5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:42.511744 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad238ad-e77a-4c57-8161-f4c071215fa5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:42.524254 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-6f4d2bc7-0c1e-4533-9e61-6ca9293a5908 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cdbaac37cff41feb5441af8eb1d1c73'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:42.525092 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-6f4d2bc7-0c1e-4533-9e61-6ca9293a5908 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cdbaac37cff41feb5441af8eb1d1c73'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:42.533851 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad238ad-e77a-4c57-8161-f4c071215fa5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:42.534831 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-6f4d2bc7-0c1e-4533-9e61-6ca9293a5908 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cdbaac37cff41feb5441af8eb1d1c73'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:42.542734 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad238ad-e77a-4c57-8161-f4c071215fa5 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:42.551103 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-6f4d2bc7-0c1e-4533-9e61-6ca9293a5908 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cdbaac37cff41feb5441af8eb1d1c73'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:42.555577 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1120/2222] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:18:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:18:42.566462 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-6f4d2bc7-0c1e-4533-9e61-6ca9293a5908 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cdbaac37cff41feb5441af8eb1d1c73'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:42.574592 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1103/2223] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:18:42 2022] DELETE /identity/v3/users/7cdbaac37cff41feb5441af8eb1d1c73 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:18:42.579914 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e89d012-c876-44bc-8e60-6b87dd4e44b4 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:42.580732 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e89d012-c876-44bc-8e60-6b87dd4e44b4 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:42.580732 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e89d012-c876-44bc-8e60-6b87dd4e44b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:42.604430 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-5e89d012-c876-44bc-8e60-6b87dd4e44b4 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:18:42.605740 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5e89d012-c876-44bc-8e60-6b87dd4e44b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:42.720938 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1121/2224] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:18:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:18:42.730411 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4d2bc7-0c1e-4533-9e61-6ca9293a5908 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:42.731858 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-6f4d2bc7-0c1e-4533-9e61-6ca9293a5908 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:42.811700 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4d2bc7-0c1e-4533-9e61-6ca9293a5908 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:42.816554 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e78ab90-ab63-4852-a9f3-ebd90ebfb15e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:42.817219 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e78ab90-ab63-4852-a9f3-ebd90ebfb15e service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:42.817514 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e78ab90-ab63-4852-a9f3-ebd90ebfb15e service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:42.817857 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e78ab90-ab63-4852-a9f3-ebd90ebfb15e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:42.830994 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e78ab90-ab63-4852-a9f3-ebd90ebfb15e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:42.832841 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e78ab90-ab63-4852-a9f3-ebd90ebfb15e service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:42.852563 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1104/2225] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:18:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:18:43.212323 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e89d012-c876-44bc-8e60-6b87dd4e44b4 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:43.214026 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5e89d012-c876-44bc-8e60-6b87dd4e44b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:43.227230 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e89d012-c876-44bc-8e60-6b87dd4e44b4 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:43.232878 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374e50cc-98cc-467d-b056-892f699c44bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:43.233525 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e50cc-98cc-467d-b056-892f699c44bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:43.233804 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e50cc-98cc-467d-b056-892f699c44bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:43.234075 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e50cc-98cc-467d-b056-892f699c44bc admin admin] PATH_INFO: `/v3/projects/876fa8cb471c4f06bf12518e0439f384` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:43.235233 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374e50cc-98cc-467d-b056-892f699c44bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=876fa8cb471c4f06bf12518e0439f384)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:43.237797 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374e50cc-98cc-467d-b056-892f699c44bc admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:43.294594 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-374e50cc-98cc-467d-b056-892f699c44bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '876fa8cb471c4f06bf12518e0439f384', 'request_id': 'req-374e50cc-98cc-467d-b056-892f699c44bc'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:43.300543 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-374e50cc-98cc-467d-b056-892f699c44bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '876fa8cb471c4f06bf12518e0439f384', 'request_id': 'req-374e50cc-98cc-467d-b056-892f699c44bc'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:43.310184 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-374e50cc-98cc-467d-b056-892f699c44bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '876fa8cb471c4f06bf12518e0439f384', 'request_id': 'req-374e50cc-98cc-467d-b056-892f699c44bc'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:43.322163 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-374e50cc-98cc-467d-b056-892f699c44bc admin admin] The token cache is being invalidate because project 876fa8cb471c4f06bf12518e0439f384 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:18:43.322448 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-374e50cc-98cc-467d-b056-892f699c44bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:43.323111 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-374e50cc-98cc-467d-b056-892f699c44bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:43.324708 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1122/2226] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:18:43 2022] DELETE /identity/v3/projects/876fa8cb471c4f06bf12518e0439f384 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:18:43.328982 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e78ab90-ab63-4852-a9f3-ebd90ebfb15e service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:43.339452 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e78ab90-ab63-4852-a9f3-ebd90ebfb15e service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:43.387029 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:18:43.400309 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b06ee6-cd3b-47f6-9895-48da9e9f9218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:43.400976 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b06ee6-cd3b-47f6-9895-48da9e9f9218 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:43.401250 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b06ee6-cd3b-47f6-9895-48da9e9f9218 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:43.401512 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b06ee6-cd3b-47f6-9895-48da9e9f9218 admin admin] PATH_INFO: `/v3/users/c2ff2240cbb14d2f85253849fff8e10b` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:43.402640 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b06ee6-cd3b-47f6-9895-48da9e9f9218 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2ff2240cbb14d2f85253849fff8e10b)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:43.404561 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b06ee6-cd3b-47f6-9895-48da9e9f9218 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:43.456354 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d8b06ee6-cd3b-47f6-9895-48da9e9f9218 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2ff2240cbb14d2f85253849fff8e10b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:43.457147 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d8b06ee6-cd3b-47f6-9895-48da9e9f9218 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2ff2240cbb14d2f85253849fff8e10b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:43.464767 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d8b06ee6-cd3b-47f6-9895-48da9e9f9218 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2ff2240cbb14d2f85253849fff8e10b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:43.477882 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d8b06ee6-cd3b-47f6-9895-48da9e9f9218 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2ff2240cbb14d2f85253849fff8e10b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:43.486134 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d8b06ee6-cd3b-47f6-9895-48da9e9f9218 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2ff2240cbb14d2f85253849fff8e10b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:43.495337 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1105/2227] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:18:43 2022] DELETE /identity/v3/users/c2ff2240cbb14d2f85253849fff8e10b => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:18:43.947084 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374e50cc-98cc-467d-b056-892f699c44bc admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:43.949163 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-374e50cc-98cc-467d-b056-892f699c44bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:43.964058 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374e50cc-98cc-467d-b056-892f699c44bc admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:43.973746 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:18:44.037339 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc64a892-7814-4727-a479-645fa560a345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:44.037994 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc64a892-7814-4727-a479-645fa560a345 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:44.038268 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc64a892-7814-4727-a479-645fa560a345 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:44.038541 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc64a892-7814-4727-a479-645fa560a345 admin admin] PATH_INFO: `/v3/projects/f9d32e6dfaf3452e95b0c67d34f6574d` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:44.039994 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc64a892-7814-4727-a479-645fa560a345 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9d32e6dfaf3452e95b0c67d34f6574d)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:44.042000 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc64a892-7814-4727-a479-645fa560a345 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:44.102149 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-fc64a892-7814-4727-a479-645fa560a345 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9d32e6dfaf3452e95b0c67d34f6574d', 'request_id': 'req-fc64a892-7814-4727-a479-645fa560a345'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:44.109395 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-fc64a892-7814-4727-a479-645fa560a345 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9d32e6dfaf3452e95b0c67d34f6574d', 'request_id': 'req-fc64a892-7814-4727-a479-645fa560a345'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:44.118568 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-fc64a892-7814-4727-a479-645fa560a345 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9d32e6dfaf3452e95b0c67d34f6574d', 'request_id': 'req-fc64a892-7814-4727-a479-645fa560a345'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:44.130931 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-fc64a892-7814-4727-a479-645fa560a345 admin admin] The token cache is being invalidate because project f9d32e6dfaf3452e95b0c67d34f6574d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:18:44.131210 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-fc64a892-7814-4727-a479-645fa560a345 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:44.131973 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-fc64a892-7814-4727-a479-645fa560a345 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:44.133593 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1123/2228] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:18:43 2022] DELETE /identity/v3/projects/f9d32e6dfaf3452e95b0c67d34f6574d => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:18:44.138787 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b013a47-9689-4819-b036-d552a60476bd None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:44.139341 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b013a47-9689-4819-b036-d552a60476bd None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:44.139341 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b013a47-9689-4819-b036-d552a60476bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:44.160948 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-7b013a47-9689-4819-b036-d552a60476bd None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:18:44.162235 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7b013a47-9689-4819-b036-d552a60476bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:44.232135 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1106/2229] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:18:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:18:44.240728 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53460b7-9201-48ae-bdf0-28200ad36a43 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:44.241363 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53460b7-9201-48ae-bdf0-28200ad36a43 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:44.241363 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53460b7-9201-48ae-bdf0-28200ad36a43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:44.271970 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-c53460b7-9201-48ae-bdf0-28200ad36a43 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:18:44.274006 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-c53460b7-9201-48ae-bdf0-28200ad36a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:44.286177 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1124/2230] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:18:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:18:44.292648 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b013a47-9689-4819-b036-d552a60476bd None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:44.303527 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b013a47-9689-4819-b036-d552a60476bd None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:44.310665 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a095d295-26c5-4387-9e42-f45a149b1f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:44.311278 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a095d295-26c5-4387-9e42-f45a149b1f9d admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:44.311559 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a095d295-26c5-4387-9e42-f45a149b1f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:44.311830 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a095d295-26c5-4387-9e42-f45a149b1f9d admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:44.312486 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a095d295-26c5-4387-9e42-f45a149b1f9d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:18:44.312798 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a095d295-26c5-4387-9e42-f45a149b1f9d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:44.314525 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a095d295-26c5-4387-9e42-f45a149b1f9d admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:44.327647 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1107/2231] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 6 16:18:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:18:44.333980 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53460b7-9201-48ae-bdf0-28200ad36a43 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:44.343591 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53460b7-9201-48ae-bdf0-28200ad36a43 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:44.348944 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be5720f-6cc9-4259-959b-1dc99da855b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:44.349559 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be5720f-6cc9-4259-959b-1dc99da855b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:44.349841 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be5720f-6cc9-4259-959b-1dc99da855b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:44.350116 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be5720f-6cc9-4259-959b-1dc99da855b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:44.350768 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be5720f-6cc9-4259-959b-1dc99da855b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:44.352714 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be5720f-6cc9-4259-959b-1dc99da855b4 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:44.385212 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1125/2232] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:18:44 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:18:44.390151 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a095d295-26c5-4387-9e42-f45a149b1f9d admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:44.403837 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a095d295-26c5-4387-9e42-f45a149b1f9d admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:44.462023 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:18:44.470139 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6461f8a0-c7f8-4a37-adc7-23e860d49be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:44.470775 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6461f8a0-c7f8-4a37-adc7-23e860d49be4 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:44.471106 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6461f8a0-c7f8-4a37-adc7-23e860d49be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:44.471387 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6461f8a0-c7f8-4a37-adc7-23e860d49be4 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:44.472038 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6461f8a0-c7f8-4a37-adc7-23e860d49be4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:44.474246 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6461f8a0-c7f8-4a37-adc7-23e860d49be4 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:44.498110 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1108/2233] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:18:44 2022] POST /identity/v3/users => generated 292 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:18:44.502420 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be5720f-6cc9-4259-959b-1dc99da855b4 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:44.513195 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be5720f-6cc9-4259-959b-1dc99da855b4 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:44.518948 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cafc2bc-45dc-4d97-812c-c68516dd342c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:44.519544 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cafc2bc-45dc-4d97-812c-c68516dd342c admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:44.519816 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cafc2bc-45dc-4d97-812c-c68516dd342c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:44.520082 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cafc2bc-45dc-4d97-812c-c68516dd342c admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:44.520723 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cafc2bc-45dc-4d97-812c-c68516dd342c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:44.522353 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cafc2bc-45dc-4d97-812c-c68516dd342c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:44.532316 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1126/2234] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:18:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:18:44.537978 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6461f8a0-c7f8-4a37-adc7-23e860d49be4 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:44.548433 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6461f8a0-c7f8-4a37-adc7-23e860d49be4 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:44.554385 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3435045f-d007-43a5-bf5a-2361898ddcb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:44.554975 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3435045f-d007-43a5-bf5a-2361898ddcb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:44.555257 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3435045f-d007-43a5-bf5a-2361898ddcb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:44.555527 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3435045f-d007-43a5-bf5a-2361898ddcb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:44.556167 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3435045f-d007-43a5-bf5a-2361898ddcb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:44.557812 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3435045f-d007-43a5-bf5a-2361898ddcb0 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:44.569421 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1109/2235] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:18:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:18:44.574047 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cafc2bc-45dc-4d97-812c-c68516dd342c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:44.586256 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cafc2bc-45dc-4d97-812c-c68516dd342c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:44.592217 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2d7e0c-cd73-4169-9ed4-0a4b7578bf34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:44.592933 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d7e0c-cd73-4169-9ed4-0a4b7578bf34 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:44.593208 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d7e0c-cd73-4169-9ed4-0a4b7578bf34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:44.593472 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d7e0c-cd73-4169-9ed4-0a4b7578bf34 admin admin] PATH_INFO: `/v3/projects/25043d61db0c4ffc98c989bddde8856a/users/d2719ef751664c059e4e267ba47fcc84/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:44.610129 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2d7e0c-cd73-4169-9ed4-0a4b7578bf34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25043d61db0c4ffc98c989bddde8856a, user_id=d2719ef751664c059e4e267ba47fcc84, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:44.612949 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2d7e0c-cd73-4169-9ed4-0a4b7578bf34 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:44.629219 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1127/2236] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:18:44 2022] PUT /identity/v3/projects/25043d61db0c4ffc98c989bddde8856a/users/d2719ef751664c059e4e267ba47fcc84/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:18:44.641970 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165f2187-1d97-4ea5-adbc-52345d7b48b4 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:44.642267 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165f2187-1d97-4ea5-adbc-52345d7b48b4 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:44.642539 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165f2187-1d97-4ea5-adbc-52345d7b48b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:44.672372 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-165f2187-1d97-4ea5-adbc-52345d7b48b4 None None] MFA Rules not processed for user `d2719ef751664c059e4e267ba47fcc84`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:18:44.673746 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-165f2187-1d97-4ea5-adbc-52345d7b48b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:44.746674 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1110/2237] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:18:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:18:44.747730 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2d7e0c-cd73-4169-9ed4-0a4b7578bf34 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:44.804456 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:18:44.834768 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2d7e0c-cd73-4169-9ed4-0a4b7578bf34 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:44.839816 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3255b4-4e3b-48ee-b0a3-bee920609766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:44.840702 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3255b4-4e3b-48ee-b0a3-bee920609766 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:44.840993 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3255b4-4e3b-48ee-b0a3-bee920609766 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:44.841246 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3255b4-4e3b-48ee-b0a3-bee920609766 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:44.841872 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3255b4-4e3b-48ee-b0a3-bee920609766 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:44.844042 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3255b4-4e3b-48ee-b0a3-bee920609766 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:44.889810 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1128/2238] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:18:44 2022] POST /identity/v3/projects => generated 383 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:18:44.895039 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165f2187-1d97-4ea5-adbc-52345d7b48b4 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:44.906975 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165f2187-1d97-4ea5-adbc-52345d7b48b4 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:44.972734 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1679b449-9ddb-43ce-8e7a-fa500eeecf48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:44.973378 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1679b449-9ddb-43ce-8e7a-fa500eeecf48 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:44.973655 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1679b449-9ddb-43ce-8e7a-fa500eeecf48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:44.973924 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1679b449-9ddb-43ce-8e7a-fa500eeecf48 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:44.974631 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1679b449-9ddb-43ce-8e7a-fa500eeecf48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:44.976540 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1679b449-9ddb-43ce-8e7a-fa500eeecf48 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:45.006936 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1111/2239] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:18:44 2022] POST /identity/v3/users => generated 298 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:18:45.010888 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3255b4-4e3b-48ee-b0a3-bee920609766 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:45.023076 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3255b4-4e3b-48ee-b0a3-bee920609766 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:45.029386 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efafa86d-2515-4eb4-9bb7-23d71d0efc97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:45.030464 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efafa86d-2515-4eb4-9bb7-23d71d0efc97 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:45.030949 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efafa86d-2515-4eb4-9bb7-23d71d0efc97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:45.031407 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efafa86d-2515-4eb4-9bb7-23d71d0efc97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:45.032617 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efafa86d-2515-4eb4-9bb7-23d71d0efc97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:45.035542 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efafa86d-2515-4eb4-9bb7-23d71d0efc97 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:45.060555 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1129/2240] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:18:45 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:18:45.062462 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1679b449-9ddb-43ce-8e7a-fa500eeecf48 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:45.081935 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1679b449-9ddb-43ce-8e7a-fa500eeecf48 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:45.089360 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8568e428-34fc-474c-b02f-4350b10bd913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:45.090063 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568e428-34fc-474c-b02f-4350b10bd913 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:45.090339 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568e428-34fc-474c-b02f-4350b10bd913 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:45.090614 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568e428-34fc-474c-b02f-4350b10bd913 admin admin] PATH_INFO: `/v3/projects/3e7bb2836e6d4040b24c92d7d83df39f/users/9ee9b945cd3a4ba091a2c3bf9d337617/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:45.110218 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8568e428-34fc-474c-b02f-4350b10bd913 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e7bb2836e6d4040b24c92d7d83df39f, user_id=9ee9b945cd3a4ba091a2c3bf9d337617, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:45.112983 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8568e428-34fc-474c-b02f-4350b10bd913 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:45.127749 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1112/2241] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:18:45 2022] PUT /identity/v3/projects/3e7bb2836e6d4040b24c92d7d83df39f/users/9ee9b945cd3a4ba091a2c3bf9d337617/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:18:45.133288 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efafa86d-2515-4eb4-9bb7-23d71d0efc97 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:45.155430 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efafa86d-2515-4eb4-9bb7-23d71d0efc97 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:45.225339 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c38dd98-5c45-4f0b-8016-2e52642d7125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:45.226074 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c38dd98-5c45-4f0b-8016-2e52642d7125 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:45.226225 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c38dd98-5c45-4f0b-8016-2e52642d7125 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:45.226511 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c38dd98-5c45-4f0b-8016-2e52642d7125 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:45.227538 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c38dd98-5c45-4f0b-8016-2e52642d7125 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:45.230021 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c38dd98-5c45-4f0b-8016-2e52642d7125 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:45.246304 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1130/2242] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:18:45 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:18:45.250021 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8568e428-34fc-474c-b02f-4350b10bd913 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:45.262112 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8568e428-34fc-474c-b02f-4350b10bd913 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:45.267225 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d82358b-0015-406f-950e-70c9eceeaa07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:45.267925 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d82358b-0015-406f-950e-70c9eceeaa07 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:45.268202 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d82358b-0015-406f-950e-70c9eceeaa07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:45.268485 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d82358b-0015-406f-950e-70c9eceeaa07 admin admin] PATH_INFO: `/v3/projects/3e7bb2836e6d4040b24c92d7d83df39f/users/9ee9b945cd3a4ba091a2c3bf9d337617/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:45.270668 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d82358b-0015-406f-950e-70c9eceeaa07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e7bb2836e6d4040b24c92d7d83df39f, user_id=9ee9b945cd3a4ba091a2c3bf9d337617, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:45.276155 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d82358b-0015-406f-950e-70c9eceeaa07 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:45.286574 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1113/2243] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:18:45 2022] PUT /identity/v3/projects/3e7bb2836e6d4040b24c92d7d83df39f/users/9ee9b945cd3a4ba091a2c3bf9d337617/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:18:45.301253 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb5328b-1dbc-443d-8da7-e2104c615269 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:45.301738 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb5328b-1dbc-443d-8da7-e2104c615269 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:45.302196 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb5328b-1dbc-443d-8da7-e2104c615269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:45.350294 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-2cb5328b-1dbc-443d-8da7-e2104c615269 None None] MFA Rules not processed for user `9ee9b945cd3a4ba091a2c3bf9d337617`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:18:45.351892 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-2cb5328b-1dbc-443d-8da7-e2104c615269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:45.442521 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1131/2244] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:18:45 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:18:45.452007 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d82358b-0015-406f-950e-70c9eceeaa07 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:45.453890 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-2d82358b-0015-406f-950e-70c9eceeaa07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:45.474691 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:18:45.516921 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d82358b-0015-406f-950e-70c9eceeaa07 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:45.520643 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73792519-f305-479f-95d9-8762fb4556cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:45.521289 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73792519-f305-479f-95d9-8762fb4556cb service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:45.521567 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73792519-f305-479f-95d9-8762fb4556cb service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:45.521920 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73792519-f305-479f-95d9-8762fb4556cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:45.532000 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73792519-f305-479f-95d9-8762fb4556cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:45.533715 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73792519-f305-479f-95d9-8762fb4556cb service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:45.547004 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1114/2245] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:18:45 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:18:45.666662 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb5328b-1dbc-443d-8da7-e2104c615269 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:45.668415 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-2cb5328b-1dbc-443d-8da7-e2104c615269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:45.680730 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb5328b-1dbc-443d-8da7-e2104c615269 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:45.684286 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefebb70-664e-4ea2-a4c2-725dd978b4e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:45.684939 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefebb70-664e-4ea2-a4c2-725dd978b4e0 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:45.685214 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefebb70-664e-4ea2-a4c2-725dd978b4e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:45.685788 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefebb70-664e-4ea2-a4c2-725dd978b4e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:45.724037 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:18:45.757550 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefebb70-664e-4ea2-a4c2-725dd978b4e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:45.759022 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefebb70-664e-4ea2-a4c2-725dd978b4e0 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:45.768919 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1132/2246] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:18:45 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:18:48.511937 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73792519-f305-479f-95d9-8762fb4556cb service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:48.590846 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73792519-f305-479f-95d9-8762fb4556cb service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:48.600333 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8844823f-d70f-40e7-ae62-433c1f1deac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:48.601467 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8844823f-d70f-40e7-ae62-433c1f1deac7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:48.601876 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8844823f-d70f-40e7-ae62-433c1f1deac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:48.602204 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8844823f-d70f-40e7-ae62-433c1f1deac7 admin admin] PATH_INFO: `/v3/users/d2719ef751664c059e4e267ba47fcc84` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:48.604047 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8844823f-d70f-40e7-ae62-433c1f1deac7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2719ef751664c059e4e267ba47fcc84)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:48.606356 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8844823f-d70f-40e7-ae62-433c1f1deac7 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:48.678712 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8844823f-d70f-40e7-ae62-433c1f1deac7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2719ef751664c059e4e267ba47fcc84'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:48.679442 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8844823f-d70f-40e7-ae62-433c1f1deac7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2719ef751664c059e4e267ba47fcc84'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:48.687877 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8844823f-d70f-40e7-ae62-433c1f1deac7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2719ef751664c059e4e267ba47fcc84'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:48.701061 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8844823f-d70f-40e7-ae62-433c1f1deac7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2719ef751664c059e4e267ba47fcc84'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:48.711550 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8844823f-d70f-40e7-ae62-433c1f1deac7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2719ef751664c059e4e267ba47fcc84'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:48.719140 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1115/2247] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:18:48 2022] DELETE /identity/v3/users/d2719ef751664c059e4e267ba47fcc84 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:18:48.724581 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc0d78e-4d64-4160-9f9e-c70b7daefc3a None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:48.725347 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc0d78e-4d64-4160-9f9e-c70b7daefc3a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:48.725495 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc0d78e-4d64-4160-9f9e-c70b7daefc3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:48.749389 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-4fc0d78e-4d64-4160-9f9e-c70b7daefc3a None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:18:48.750499 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-4fc0d78e-4d64-4160-9f9e-c70b7daefc3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:48.827616 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1133/2248] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:18:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:18:48.848342 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8844823f-d70f-40e7-ae62-433c1f1deac7 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:48.850771 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8844823f-d70f-40e7-ae62-433c1f1deac7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:48.903978 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:18:48.923165 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8844823f-d70f-40e7-ae62-433c1f1deac7 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:48.928200 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7ea4d0-7dd3-4729-9839-f1c8223cda10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:48.928867 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7ea4d0-7dd3-4729-9839-f1c8223cda10 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:48.929149 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7ea4d0-7dd3-4729-9839-f1c8223cda10 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:48.929415 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7ea4d0-7dd3-4729-9839-f1c8223cda10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:48.940998 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7ea4d0-7dd3-4729-9839-f1c8223cda10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:48.942435 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7ea4d0-7dd3-4729-9839-f1c8223cda10 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:48.953139 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1116/2249] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:18:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:18:49.266879 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc0d78e-4d64-4160-9f9e-c70b7daefc3a None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:49.269187 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-4fc0d78e-4d64-4160-9f9e-c70b7daefc3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:49.289870 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc0d78e-4d64-4160-9f9e-c70b7daefc3a None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:49.295696 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1990454b-f258-4185-bbd6-36ca5b6da01c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:49.296338 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1990454b-f258-4185-bbd6-36ca5b6da01c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:49.296614 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1990454b-f258-4185-bbd6-36ca5b6da01c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:49.296923 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1990454b-f258-4185-bbd6-36ca5b6da01c admin admin] PATH_INFO: `/v3/projects/25043d61db0c4ffc98c989bddde8856a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:49.298054 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1990454b-f258-4185-bbd6-36ca5b6da01c admin admin] RBAC: Authorizing `identity:delete_project(project_id=25043d61db0c4ffc98c989bddde8856a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:49.300072 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1990454b-f258-4185-bbd6-36ca5b6da01c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:49.350746 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-1990454b-f258-4185-bbd6-36ca5b6da01c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25043d61db0c4ffc98c989bddde8856a', 'request_id': 'req-1990454b-f258-4185-bbd6-36ca5b6da01c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:49.356317 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-1990454b-f258-4185-bbd6-36ca5b6da01c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25043d61db0c4ffc98c989bddde8856a', 'request_id': 'req-1990454b-f258-4185-bbd6-36ca5b6da01c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:49.364989 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-1990454b-f258-4185-bbd6-36ca5b6da01c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25043d61db0c4ffc98c989bddde8856a', 'request_id': 'req-1990454b-f258-4185-bbd6-36ca5b6da01c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:49.383877 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-1990454b-f258-4185-bbd6-36ca5b6da01c admin admin] The token cache is being invalidate because project 25043d61db0c4ffc98c989bddde8856a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:18:49.384201 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-1990454b-f258-4185-bbd6-36ca5b6da01c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:49.384830 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-1990454b-f258-4185-bbd6-36ca5b6da01c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:49.386342 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1134/2250] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:18:49 2022] DELETE /identity/v3/projects/25043d61db0c4ffc98c989bddde8856a => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:18:49.390575 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7ea4d0-7dd3-4729-9839-f1c8223cda10 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:49.400072 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7ea4d0-7dd3-4729-9839-f1c8223cda10 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:49.469799 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225dd292-92c9-4aa6-bae4-8f37a5180001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:49.470148 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225dd292-92c9-4aa6-bae4-8f37a5180001 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:49.470554 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225dd292-92c9-4aa6-bae4-8f37a5180001 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:49.470905 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225dd292-92c9-4aa6-bae4-8f37a5180001 admin admin] PATH_INFO: `/v3/users/9ee9b945cd3a4ba091a2c3bf9d337617` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:49.473323 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225dd292-92c9-4aa6-bae4-8f37a5180001 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ee9b945cd3a4ba091a2c3bf9d337617)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:49.475775 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225dd292-92c9-4aa6-bae4-8f37a5180001 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:49.529717 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-225dd292-92c9-4aa6-bae4-8f37a5180001 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ee9b945cd3a4ba091a2c3bf9d337617'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:49.530436 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-225dd292-92c9-4aa6-bae4-8f37a5180001 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ee9b945cd3a4ba091a2c3bf9d337617'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:49.538439 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-225dd292-92c9-4aa6-bae4-8f37a5180001 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ee9b945cd3a4ba091a2c3bf9d337617'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:49.557645 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-225dd292-92c9-4aa6-bae4-8f37a5180001 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ee9b945cd3a4ba091a2c3bf9d337617'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:49.569374 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-225dd292-92c9-4aa6-bae4-8f37a5180001 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ee9b945cd3a4ba091a2c3bf9d337617'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:49.576319 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1117/2251] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:18:49 2022] DELETE /identity/v3/users/9ee9b945cd3a4ba091a2c3bf9d337617 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:18:50.053620 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1990454b-f258-4185-bbd6-36ca5b6da01c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:50.055340 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1990454b-f258-4185-bbd6-36ca5b6da01c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:50.074055 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1990454b-f258-4185-bbd6-36ca5b6da01c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:50.130336 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:18:50.138003 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91034040-d5cc-4756-a9c4-63275f4c812b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:50.139038 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91034040-d5cc-4756-a9c4-63275f4c812b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:50.139527 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91034040-d5cc-4756-a9c4-63275f4c812b admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:50.139842 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91034040-d5cc-4756-a9c4-63275f4c812b admin admin] PATH_INFO: `/v3/projects/3e7bb2836e6d4040b24c92d7d83df39f` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:50.141549 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91034040-d5cc-4756-a9c4-63275f4c812b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e7bb2836e6d4040b24c92d7d83df39f)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:50.144178 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91034040-d5cc-4756-a9c4-63275f4c812b admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:50.204663 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-91034040-d5cc-4756-a9c4-63275f4c812b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e7bb2836e6d4040b24c92d7d83df39f', 'request_id': 'req-91034040-d5cc-4756-a9c4-63275f4c812b'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:50.211427 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-91034040-d5cc-4756-a9c4-63275f4c812b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e7bb2836e6d4040b24c92d7d83df39f', 'request_id': 'req-91034040-d5cc-4756-a9c4-63275f4c812b'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:50.220545 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-91034040-d5cc-4756-a9c4-63275f4c812b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e7bb2836e6d4040b24c92d7d83df39f', 'request_id': 'req-91034040-d5cc-4756-a9c4-63275f4c812b'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:50.233581 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-91034040-d5cc-4756-a9c4-63275f4c812b admin admin] The token cache is being invalidate because project 3e7bb2836e6d4040b24c92d7d83df39f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:18:50.233973 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-91034040-d5cc-4756-a9c4-63275f4c812b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:50.234722 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-91034040-d5cc-4756-a9c4-63275f4c812b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:50.236382 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1135/2252] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:18:50 2022] DELETE /identity/v3/projects/3e7bb2836e6d4040b24c92d7d83df39f => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:18:50.241510 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38fcfc1-9451-492f-b568-e44fa7960638 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:50.241895 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38fcfc1-9451-492f-b568-e44fa7960638 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:50.242166 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38fcfc1-9451-492f-b568-e44fa7960638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:50.268881 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-d38fcfc1-9451-492f-b568-e44fa7960638 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:18:50.270163 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d38fcfc1-9451-492f-b568-e44fa7960638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:50.341147 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:18:50.355062 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1118/2253] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:18:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:18:50.358641 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eac6ef3-2107-4118-b136-c425519e17f5 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:50.358929 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eac6ef3-2107-4118-b136-c425519e17f5 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:50.359245 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eac6ef3-2107-4118-b136-c425519e17f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:50.383386 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-6eac6ef3-2107-4118-b136-c425519e17f5 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:18:50.384884 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6eac6ef3-2107-4118-b136-c425519e17f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:50.398082 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1136/2254] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:18:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:18:50.402712 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38fcfc1-9451-492f-b568-e44fa7960638 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:50.420324 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38fcfc1-9451-492f-b568-e44fa7960638 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:50.426461 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42738b6-da18-4c17-83a5-632449b732ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:50.427113 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42738b6-da18-4c17-83a5-632449b732ea admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:50.427388 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42738b6-da18-4c17-83a5-632449b732ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:50.427659 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42738b6-da18-4c17-83a5-632449b732ea admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:50.428335 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42738b6-da18-4c17-83a5-632449b732ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:18:50.428637 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42738b6-da18-4c17-83a5-632449b732ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:50.430381 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42738b6-da18-4c17-83a5-632449b732ea admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:50.447957 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1119/2255] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 6 16:18:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:18:50.454203 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eac6ef3-2107-4118-b136-c425519e17f5 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:50.466962 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eac6ef3-2107-4118-b136-c425519e17f5 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:50.478187 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5718435-ea2c-41aa-8e50-6c1ab50c65f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:50.479383 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5718435-ea2c-41aa-8e50-6c1ab50c65f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:50.480473 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5718435-ea2c-41aa-8e50-6c1ab50c65f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:50.481916 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5718435-ea2c-41aa-8e50-6c1ab50c65f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:50.484501 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5718435-ea2c-41aa-8e50-6c1ab50c65f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:50.487896 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5718435-ea2c-41aa-8e50-6c1ab50c65f2 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:50.531751 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1137/2256] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:18:50 2022] POST /identity/v3/projects => generated 379 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:18:50.536207 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42738b6-da18-4c17-83a5-632449b732ea admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:50.547535 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42738b6-da18-4c17-83a5-632449b732ea admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:50.614389 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3baa60d-914a-45b5-90c0-47ec18f8bf21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:50.615249 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3baa60d-914a-45b5-90c0-47ec18f8bf21 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:50.615598 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3baa60d-914a-45b5-90c0-47ec18f8bf21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:50.617433 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3baa60d-914a-45b5-90c0-47ec18f8bf21 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:50.617433 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3baa60d-914a-45b5-90c0-47ec18f8bf21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:50.619613 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3baa60d-914a-45b5-90c0-47ec18f8bf21 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:50.654528 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1120/2257] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:18:50 2022] POST /identity/v3/users => generated 290 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:18:50.660920 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5718435-ea2c-41aa-8e50-6c1ab50c65f2 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:50.688933 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5718435-ea2c-41aa-8e50-6c1ab50c65f2 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:50.688933 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79026a68-6800-436f-bc37-e5d8cd9e3d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:50.688933 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79026a68-6800-436f-bc37-e5d8cd9e3d0b admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:50.688933 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79026a68-6800-436f-bc37-e5d8cd9e3d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:50.689692 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79026a68-6800-436f-bc37-e5d8cd9e3d0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:50.689692 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79026a68-6800-436f-bc37-e5d8cd9e3d0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:50.689839 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79026a68-6800-436f-bc37-e5d8cd9e3d0b admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:50.698623 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1832628-269d-4a1d-a60f-cb858be6711c None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:50.698940 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1832628-269d-4a1d-a60f-cb858be6711c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:50.699220 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1832628-269d-4a1d-a60f-cb858be6711c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:50.716567 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1138/2258] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:18:50 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:18:50.721324 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79026a68-6800-436f-bc37-e5d8cd9e3d0b admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:50.733484 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-e1832628-269d-4a1d-a60f-cb858be6711c None None] MFA Rules not processed for user `2921386cc370487da26a786c8198c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:18:50.734784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e1832628-269d-4a1d-a60f-cb858be6711c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:50.735081 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79026a68-6800-436f-bc37-e5d8cd9e3d0b admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:50.739998 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc650a8c-41a6-4782-989d-d4edd67e0a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:50.740621 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc650a8c-41a6-4782-989d-d4edd67e0a34 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:50.740931 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc650a8c-41a6-4782-989d-d4edd67e0a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:50.741205 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc650a8c-41a6-4782-989d-d4edd67e0a34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:50.741862 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc650a8c-41a6-4782-989d-d4edd67e0a34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:50.743510 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc650a8c-41a6-4782-989d-d4edd67e0a34 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:50.757822 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1139/2259] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:18:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 16:18:50.759550 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc650a8c-41a6-4782-989d-d4edd67e0a34 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:50.770019 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc650a8c-41a6-4782-989d-d4edd67e0a34 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:50.776111 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d922a9-b228-485d-914f-52bb1bb401ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:50.776821 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d922a9-b228-485d-914f-52bb1bb401ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:50.777103 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d922a9-b228-485d-914f-52bb1bb401ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:50.777374 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d922a9-b228-485d-914f-52bb1bb401ee admin admin] PATH_INFO: `/v3/projects/ad3bd3aa92c24f3f9324d0faf40e3292/users/46e4b8ae0a3646839edee06fe671cf04/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:50.795674 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d922a9-b228-485d-914f-52bb1bb401ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad3bd3aa92c24f3f9324d0faf40e3292, user_id=46e4b8ae0a3646839edee06fe671cf04, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:50.798905 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d922a9-b228-485d-914f-52bb1bb401ee admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:50.815568 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1121/2261] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:18:50 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:18:50.816843 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1140/2261] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:18:50 2022] PUT /identity/v3/projects/ad3bd3aa92c24f3f9324d0faf40e3292/users/46e4b8ae0a3646839edee06fe671cf04/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:18:50.827454 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc470c7-576c-4768-b9c8-20613f1e3d84 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:50.827822 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc470c7-576c-4768-b9c8-20613f1e3d84 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:50.828608 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc470c7-576c-4768-b9c8-20613f1e3d84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:50.859122 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-ffc470c7-576c-4768-b9c8-20613f1e3d84 None None] MFA Rules not processed for user `46e4b8ae0a3646839edee06fe671cf04`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:18:50.860420 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ffc470c7-576c-4768-b9c8-20613f1e3d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:50.941576 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1122/2262] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:18:50 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:18:50.946239 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d922a9-b228-485d-914f-52bb1bb401ee admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:51.072638 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d922a9-b228-485d-914f-52bb1bb401ee admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:51.078048 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718956ac-debd-4e15-9881-f52026cabc09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:51.078834 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718956ac-debd-4e15-9881-f52026cabc09 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:51.079142 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718956ac-debd-4e15-9881-f52026cabc09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:51.079507 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718956ac-debd-4e15-9881-f52026cabc09 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:51.080342 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718956ac-debd-4e15-9881-f52026cabc09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:51.082444 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718956ac-debd-4e15-9881-f52026cabc09 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:51.108161 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:18:51.124837 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1141/2263] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:18:50 2022] POST /identity/v3/projects => generated 379 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:18:51.127055 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc470c7-576c-4768-b9c8-20613f1e3d84 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:51.138638 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc470c7-576c-4768-b9c8-20613f1e3d84 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:51.149631 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:18:51.205914 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f176b95-294e-46c7-88ba-7610a07b2272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:51.206653 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f176b95-294e-46c7-88ba-7610a07b2272 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:51.206997 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f176b95-294e-46c7-88ba-7610a07b2272 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:51.207337 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f176b95-294e-46c7-88ba-7610a07b2272 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:51.208641 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f176b95-294e-46c7-88ba-7610a07b2272 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:51.210264 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f176b95-294e-46c7-88ba-7610a07b2272 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:51.242398 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1123/2264] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:18:51 2022] POST /identity/v3/users => generated 290 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:18:51.246820 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718956ac-debd-4e15-9881-f52026cabc09 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:51.262656 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718956ac-debd-4e15-9881-f52026cabc09 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:51.270663 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eb45cb-771d-4cb2-ab25-c9b381c78644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:51.279170 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eb45cb-771d-4cb2-ab25-c9b381c78644 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:51.280576 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eb45cb-771d-4cb2-ab25-c9b381c78644 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:51.281574 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eb45cb-771d-4cb2-ab25-c9b381c78644 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:51.282957 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eb45cb-771d-4cb2-ab25-c9b381c78644 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:51.285383 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eb45cb-771d-4cb2-ab25-c9b381c78644 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:51.303906 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1142/2265] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:18:51 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:18:51.307504 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f176b95-294e-46c7-88ba-7610a07b2272 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:51.330497 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f176b95-294e-46c7-88ba-7610a07b2272 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:51.335715 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90965917-c83d-4bb9-9e15-8e4c61d6f432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:51.336345 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90965917-c83d-4bb9-9e15-8e4c61d6f432 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:51.336671 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90965917-c83d-4bb9-9e15-8e4c61d6f432 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:51.336983 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90965917-c83d-4bb9-9e15-8e4c61d6f432 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:51.337641 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90965917-c83d-4bb9-9e15-8e4c61d6f432 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:51.339478 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90965917-c83d-4bb9-9e15-8e4c61d6f432 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:51.368802 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1124/2266] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:18:51 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:18:51.372706 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eb45cb-771d-4cb2-ab25-c9b381c78644 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:51.387782 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eb45cb-771d-4cb2-ab25-c9b381c78644 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:51.398268 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a772e82-c319-4de0-81d7-880104ca3e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:51.399261 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a772e82-c319-4de0-81d7-880104ca3e07 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:51.399800 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a772e82-c319-4de0-81d7-880104ca3e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:51.400260 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a772e82-c319-4de0-81d7-880104ca3e07 admin admin] PATH_INFO: `/v3/projects/d4801e26a7a94835a7f170babc10946b/users/d3b215a77642439e8d1fdb8a13bed2b5/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:51.419565 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a772e82-c319-4de0-81d7-880104ca3e07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4801e26a7a94835a7f170babc10946b, user_id=d3b215a77642439e8d1fdb8a13bed2b5, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:51.422759 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a772e82-c319-4de0-81d7-880104ca3e07 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:51.452871 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1143/2267] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:18:51 2022] PUT /identity/v3/projects/d4801e26a7a94835a7f170babc10946b/users/d3b215a77642439e8d1fdb8a13bed2b5/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:18:51.464024 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5a8e20-ff79-4c2f-9bb3-265431ddc1a4 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:51.464526 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5a8e20-ff79-4c2f-9bb3-265431ddc1a4 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:51.465114 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5a8e20-ff79-4c2f-9bb3-265431ddc1a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:51.499117 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-2a5a8e20-ff79-4c2f-9bb3-265431ddc1a4 None None] MFA Rules not processed for user `d3b215a77642439e8d1fdb8a13bed2b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:18:51.500369 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-2a5a8e20-ff79-4c2f-9bb3-265431ddc1a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:51.605403 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1125/2268] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:18:51 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:18:51.610680 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a772e82-c319-4de0-81d7-880104ca3e07 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:51.705863 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a772e82-c319-4de0-81d7-880104ca3e07 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:51.711126 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8583bf63-7718-47ea-8e36-5bd8e933187a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:51.711752 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8583bf63-7718-47ea-8e36-5bd8e933187a admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:51.712030 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8583bf63-7718-47ea-8e36-5bd8e933187a admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:51.712301 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8583bf63-7718-47ea-8e36-5bd8e933187a admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:51.712978 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8583bf63-7718-47ea-8e36-5bd8e933187a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:51.715023 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8583bf63-7718-47ea-8e36-5bd8e933187a admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:51.746244 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1144/2269] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:18:51 2022] POST /identity/v3/projects => generated 379 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:18:51.747865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5a8e20-ff79-4c2f-9bb3-265431ddc1a4 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:51.761695 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5a8e20-ff79-4c2f-9bb3-265431ddc1a4 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:51.839347 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfa7e95-a084-4260-b030-9eb07a84db42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:51.839954 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa7e95-a084-4260-b030-9eb07a84db42 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:51.840208 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa7e95-a084-4260-b030-9eb07a84db42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:51.840496 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa7e95-a084-4260-b030-9eb07a84db42 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:51.841167 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfa7e95-a084-4260-b030-9eb07a84db42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:51.843134 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfa7e95-a084-4260-b030-9eb07a84db42 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:51.869379 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1126/2270] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:18:51 2022] POST /identity/v3/users => generated 296 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:18:51.873777 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8583bf63-7718-47ea-8e36-5bd8e933187a admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:51.884922 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8583bf63-7718-47ea-8e36-5bd8e933187a admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:51.890136 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1c8cde-368d-4a85-aedd-f8071b4c670d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:51.890732 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1c8cde-368d-4a85-aedd-f8071b4c670d admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:51.890732 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1c8cde-368d-4a85-aedd-f8071b4c670d admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:51.891028 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1c8cde-368d-4a85-aedd-f8071b4c670d admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:51.891827 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1c8cde-368d-4a85-aedd-f8071b4c670d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:51.893982 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1c8cde-368d-4a85-aedd-f8071b4c670d admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:51.913867 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfa7e95-a084-4260-b030-9eb07a84db42 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:51.916993 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1145/2271] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:18:51 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:18:51.925255 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfa7e95-a084-4260-b030-9eb07a84db42 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:51.930366 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c21eed-6eb0-4616-a2b5-9026f4dda8dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:51.931045 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c21eed-6eb0-4616-a2b5-9026f4dda8dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:51.931347 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c21eed-6eb0-4616-a2b5-9026f4dda8dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:51.931617 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c21eed-6eb0-4616-a2b5-9026f4dda8dc admin admin] PATH_INFO: `/v3/projects/77ac8fa62fad4945a3528f618d451a8e/users/5053aeade9c54c848fd6974babed6ba0/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:51.937088 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:18:51.951633 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c21eed-6eb0-4616-a2b5-9026f4dda8dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=77ac8fa62fad4945a3528f618d451a8e, user_id=5053aeade9c54c848fd6974babed6ba0, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:51.954401 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c21eed-6eb0-4616-a2b5-9026f4dda8dc admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:51.974478 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1127/2272] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:18:51 2022] PUT /identity/v3/projects/77ac8fa62fad4945a3528f618d451a8e/users/5053aeade9c54c848fd6974babed6ba0/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:18:51.978977 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1c8cde-368d-4a85-aedd-f8071b4c670d admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:51.990078 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1c8cde-368d-4a85-aedd-f8071b4c670d admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:52.080100 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d56282-7aa8-4e8c-9a10-a531d479544a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:52.080982 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d56282-7aa8-4e8c-9a10-a531d479544a admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:52.081256 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d56282-7aa8-4e8c-9a10-a531d479544a admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:52.081601 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d56282-7aa8-4e8c-9a10-a531d479544a admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:52.082349 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d56282-7aa8-4e8c-9a10-a531d479544a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:52.084386 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d56282-7aa8-4e8c-9a10-a531d479544a admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:52.096820 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1146/2273] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:18:51 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:18:52.103964 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c21eed-6eb0-4616-a2b5-9026f4dda8dc admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:52.118626 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c21eed-6eb0-4616-a2b5-9026f4dda8dc admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:52.125621 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8d8da9-cc40-413e-807c-d52d79f13ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:52.126897 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8d8da9-cc40-413e-807c-d52d79f13ebb admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:52.127180 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8d8da9-cc40-413e-807c-d52d79f13ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:52.127460 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8d8da9-cc40-413e-807c-d52d79f13ebb admin admin] PATH_INFO: `/v3/projects/77ac8fa62fad4945a3528f618d451a8e/users/5053aeade9c54c848fd6974babed6ba0/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:52.129475 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8d8da9-cc40-413e-807c-d52d79f13ebb admin admin] RBAC: Authorizing `identity:create_grant(project_id=77ac8fa62fad4945a3528f618d451a8e, user_id=5053aeade9c54c848fd6974babed6ba0, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:52.132165 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8d8da9-cc40-413e-807c-d52d79f13ebb admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:52.145410 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1128/2274] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:18:52 2022] PUT /identity/v3/projects/77ac8fa62fad4945a3528f618d451a8e/users/5053aeade9c54c848fd6974babed6ba0/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:18:52.162658 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b9930b-9127-432a-a316-7c228802fffe None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:52.163161 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b9930b-9127-432a-a316-7c228802fffe None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:52.163505 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b9930b-9127-432a-a316-7c228802fffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:52.219582 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-44b9930b-9127-432a-a316-7c228802fffe None None] MFA Rules not processed for user `5053aeade9c54c848fd6974babed6ba0`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:18:52.220699 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-44b9930b-9127-432a-a316-7c228802fffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:52.255290 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:18:52.362311 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1147/2275] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:18:52 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:18:52.370304 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8d8da9-cc40-413e-807c-d52d79f13ebb admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:52.372691 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-1d8d8da9-cc40-413e-807c-d52d79f13ebb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:52.475471 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8d8da9-cc40-413e-807c-d52d79f13ebb admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:52.480332 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3b7218-0c55-4c82-a680-bb27f39826a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:52.481045 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b7218-0c55-4c82-a680-bb27f39826a1 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:52.481418 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b7218-0c55-4c82-a680-bb27f39826a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:52.481706 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b7218-0c55-4c82-a680-bb27f39826a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:52.491808 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3b7218-0c55-4c82-a680-bb27f39826a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:52.493314 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3b7218-0c55-4c82-a680-bb27f39826a1 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:52.504348 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1129/2276] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:18:52 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:18:52.558430 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b9930b-9127-432a-a316-7c228802fffe None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:52.560823 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-44b9930b-9127-432a-a316-7c228802fffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:52.575563 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b9930b-9127-432a-a316-7c228802fffe None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:52.592830 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36366ebd-9b50-46a1-a795-5743ef8f4b75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:52.594159 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36366ebd-9b50-46a1-a795-5743ef8f4b75 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:52.594665 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36366ebd-9b50-46a1-a795-5743ef8f4b75 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:52.595829 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36366ebd-9b50-46a1-a795-5743ef8f4b75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:52.683222 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36366ebd-9b50-46a1-a795-5743ef8f4b75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:52.686754 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36366ebd-9b50-46a1-a795-5743ef8f4b75 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:52.706931 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1148/2277] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:18:52 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:18:54.565870 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3b7218-0c55-4c82-a680-bb27f39826a1 service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:54.567810 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0e3b7218-0c55-4c82-a680-bb27f39826a1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:54.681941 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3b7218-0c55-4c82-a680-bb27f39826a1 service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:54.687536 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3413b750-51e7-4b7f-a25e-247873a8276a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:54.688181 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3413b750-51e7-4b7f-a25e-247873a8276a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:54.688926 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3413b750-51e7-4b7f-a25e-247873a8276a admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:54.689203 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3413b750-51e7-4b7f-a25e-247873a8276a admin admin] PATH_INFO: `/v3/users/04fa1b7be92a47dbb023a3bb94519323` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:54.690346 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3413b750-51e7-4b7f-a25e-247873a8276a admin admin] RBAC: Authorizing `identity:delete_user(user_id=04fa1b7be92a47dbb023a3bb94519323)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:54.692428 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3413b750-51e7-4b7f-a25e-247873a8276a admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:54.746664 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3413b750-51e7-4b7f-a25e-247873a8276a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04fa1b7be92a47dbb023a3bb94519323'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:54.747455 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3413b750-51e7-4b7f-a25e-247873a8276a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04fa1b7be92a47dbb023a3bb94519323'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:54.755505 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3413b750-51e7-4b7f-a25e-247873a8276a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04fa1b7be92a47dbb023a3bb94519323'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:54.768170 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3413b750-51e7-4b7f-a25e-247873a8276a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04fa1b7be92a47dbb023a3bb94519323'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:54.770761 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:18:54.777651 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3413b750-51e7-4b7f-a25e-247873a8276a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04fa1b7be92a47dbb023a3bb94519323'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:54.785202 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1130/2278] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:18:54 2022] DELETE /identity/v3/users/04fa1b7be92a47dbb023a3bb94519323 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:18:54.790046 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e575553-32ad-4136-b513-04ea6df3b59b None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:54.790564 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e575553-32ad-4136-b513-04ea6df3b59b None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:54.790660 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e575553-32ad-4136-b513-04ea6df3b59b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:54.814779 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-4e575553-32ad-4136-b513-04ea6df3b59b None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:18:54.816012 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-4e575553-32ad-4136-b513-04ea6df3b59b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:54.881830 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1149/2279] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:18:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:18:54.893808 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3413b750-51e7-4b7f-a25e-247873a8276a admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:54.895823 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-3413b750-51e7-4b7f-a25e-247873a8276a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:54.963409 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3413b750-51e7-4b7f-a25e-247873a8276a admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:54.967644 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1abc8bf-afe8-4758-8604-d45f296e1106 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:54.968367 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1abc8bf-afe8-4758-8604-d45f296e1106 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:54.968652 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1abc8bf-afe8-4758-8604-d45f296e1106 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:54.968953 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1abc8bf-afe8-4758-8604-d45f296e1106 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:54.980580 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1abc8bf-afe8-4758-8604-d45f296e1106 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:54.982161 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1abc8bf-afe8-4758-8604-d45f296e1106 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:54.993223 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1131/2280] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:18:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:18:55.375401 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e575553-32ad-4136-b513-04ea6df3b59b None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:55.377044 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-4e575553-32ad-4136-b513-04ea6df3b59b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:55.392219 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e575553-32ad-4136-b513-04ea6df3b59b None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:55.398601 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeae606c-307a-4f50-82f0-cd0f83bce1b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:55.399257 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae606c-307a-4f50-82f0-cd0f83bce1b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:55.399523 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae606c-307a-4f50-82f0-cd0f83bce1b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:55.399937 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae606c-307a-4f50-82f0-cd0f83bce1b9 admin admin] PATH_INFO: `/v3/projects/681a935ab92448daa87d7cab3c70d630` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:55.401122 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeae606c-307a-4f50-82f0-cd0f83bce1b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=681a935ab92448daa87d7cab3c70d630)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:55.403348 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeae606c-307a-4f50-82f0-cd0f83bce1b9 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:55.458238 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-aeae606c-307a-4f50-82f0-cd0f83bce1b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '681a935ab92448daa87d7cab3c70d630', 'request_id': 'req-aeae606c-307a-4f50-82f0-cd0f83bce1b9'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:55.460523 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:18:55.466140 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-aeae606c-307a-4f50-82f0-cd0f83bce1b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '681a935ab92448daa87d7cab3c70d630', 'request_id': 'req-aeae606c-307a-4f50-82f0-cd0f83bce1b9'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:55.482453 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-aeae606c-307a-4f50-82f0-cd0f83bce1b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '681a935ab92448daa87d7cab3c70d630', 'request_id': 'req-aeae606c-307a-4f50-82f0-cd0f83bce1b9'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:55.496166 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-aeae606c-307a-4f50-82f0-cd0f83bce1b9 admin admin] The token cache is being invalidate because project 681a935ab92448daa87d7cab3c70d630 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:18:55.496533 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-aeae606c-307a-4f50-82f0-cd0f83bce1b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:55.497267 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-aeae606c-307a-4f50-82f0-cd0f83bce1b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:55.499131 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1150/2281] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:18:55 2022] DELETE /identity/v3/projects/681a935ab92448daa87d7cab3c70d630 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:18:55.504605 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d169648c-f1a7-4c12-9d4e-615c84e2eabb None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:55.504929 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d169648c-f1a7-4c12-9d4e-615c84e2eabb None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:55.505231 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d169648c-f1a7-4c12-9d4e-615c84e2eabb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:55.526244 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-d169648c-f1a7-4c12-9d4e-615c84e2eabb None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:18:55.527495 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d169648c-f1a7-4c12-9d4e-615c84e2eabb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:55.590053 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1132/2282] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:18:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:18:55.598032 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6267ce1c-4970-4498-b4b5-adc606d32118 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:55.598393 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6267ce1c-4970-4498-b4b5-adc606d32118 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:55.599888 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6267ce1c-4970-4498-b4b5-adc606d32118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:55.621071 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-6267ce1c-4970-4498-b4b5-adc606d32118 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:18:55.622281 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6267ce1c-4970-4498-b4b5-adc606d32118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:55.630726 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1151/2283] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:18:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:18:55.635138 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d169648c-f1a7-4c12-9d4e-615c84e2eabb None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:55.645323 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d169648c-f1a7-4c12-9d4e-615c84e2eabb None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:55.650706 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513ff906-9dd6-4795-a9ab-ec135e4077c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:55.651322 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ff906-9dd6-4795-a9ab-ec135e4077c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:55.651596 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ff906-9dd6-4795-a9ab-ec135e4077c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:55.651897 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ff906-9dd6-4795-a9ab-ec135e4077c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:55.652549 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513ff906-9dd6-4795-a9ab-ec135e4077c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:18:55.654681 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513ff906-9dd6-4795-a9ab-ec135e4077c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:55.656710 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513ff906-9dd6-4795-a9ab-ec135e4077c0 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:55.669790 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1133/2284] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 6 16:18:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:18:55.675452 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6267ce1c-4970-4498-b4b5-adc606d32118 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:55.692506 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6267ce1c-4970-4498-b4b5-adc606d32118 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:55.698883 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc3da0b-b022-4e06-8f45-b54968626d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:55.699755 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc3da0b-b022-4e06-8f45-b54968626d83 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:55.700025 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc3da0b-b022-4e06-8f45-b54968626d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:55.700316 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc3da0b-b022-4e06-8f45-b54968626d83 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:55.701179 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc3da0b-b022-4e06-8f45-b54968626d83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:55.703268 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc3da0b-b022-4e06-8f45-b54968626d83 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:55.742217 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1152/2285] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:18:55 2022] POST /identity/v3/projects => generated 381 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:18:55.746862 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513ff906-9dd6-4795-a9ab-ec135e4077c0 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:55.760037 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513ff906-9dd6-4795-a9ab-ec135e4077c0 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:55.767467 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc3da0b-b022-4e06-8f45-b54968626d83 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:55.769426 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-acc3da0b-b022-4e06-8f45-b54968626d83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:55.857716 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f817152-4c81-457f-ba39-7f3f95fa284f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:55.858822 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f817152-4c81-457f-ba39-7f3f95fa284f admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:55.859227 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f817152-4c81-457f-ba39-7f3f95fa284f admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:55.859632 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f817152-4c81-457f-ba39-7f3f95fa284f admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:55.860376 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f817152-4c81-457f-ba39-7f3f95fa284f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:55.862873 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f817152-4c81-457f-ba39-7f3f95fa284f admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:55.873215 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc3da0b-b022-4e06-8f45-b54968626d83 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:55.882457 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50946759-0c4b-4eb7-8f2a-d86f8e9ee5b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:55.883657 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50946759-0c4b-4eb7-8f2a-d86f8e9ee5b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:55.883657 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50946759-0c4b-4eb7-8f2a-d86f8e9ee5b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:55.883845 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50946759-0c4b-4eb7-8f2a-d86f8e9ee5b0 admin admin] PATH_INFO: `/v3/users/46e4b8ae0a3646839edee06fe671cf04` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:55.885165 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50946759-0c4b-4eb7-8f2a-d86f8e9ee5b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46e4b8ae0a3646839edee06fe671cf04)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:55.888224 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50946759-0c4b-4eb7-8f2a-d86f8e9ee5b0 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:55.901828 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1134/2286] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:18:55 2022] POST /identity/v3/users => generated 291 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:18:55.902942 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f817152-4c81-457f-ba39-7f3f95fa284f admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:55.941543 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f817152-4c81-457f-ba39-7f3f95fa284f admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:55.948769 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b39523f-b17e-4045-b3cf-73191ac00adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:55.949868 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b39523f-b17e-4045-b3cf-73191ac00adb admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:55.950182 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b39523f-b17e-4045-b3cf-73191ac00adb admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:55.950664 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b39523f-b17e-4045-b3cf-73191ac00adb admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:55.951319 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b39523f-b17e-4045-b3cf-73191ac00adb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:55.953098 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b39523f-b17e-4045-b3cf-73191ac00adb admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:55.967176 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1135/2287] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:18:55 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:18:55.970830 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b39523f-b17e-4045-b3cf-73191ac00adb admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:55.976189 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:18:55.983155 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b39523f-b17e-4045-b3cf-73191ac00adb admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:55.983338 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-50946759-0c4b-4eb7-8f2a-d86f8e9ee5b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46e4b8ae0a3646839edee06fe671cf04'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:55.984013 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-50946759-0c4b-4eb7-8f2a-d86f8e9ee5b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46e4b8ae0a3646839edee06fe671cf04'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:55.993424 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43e519c-2761-4e75-a1b7-25d50be590ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:55.993424 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43e519c-2761-4e75-a1b7-25d50be590ff admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:55.993424 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43e519c-2761-4e75-a1b7-25d50be590ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:55.993424 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43e519c-2761-4e75-a1b7-25d50be590ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:55.993922 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43e519c-2761-4e75-a1b7-25d50be590ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:55.993922 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43e519c-2761-4e75-a1b7-25d50be590ff admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:55.999330 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-50946759-0c4b-4eb7-8f2a-d86f8e9ee5b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46e4b8ae0a3646839edee06fe671cf04'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:56.008484 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1136/2288] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:18:55 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 16:18:56.013057 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43e519c-2761-4e75-a1b7-25d50be590ff admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:56.024725 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-50946759-0c4b-4eb7-8f2a-d86f8e9ee5b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46e4b8ae0a3646839edee06fe671cf04'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:56.025502 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-50946759-0c4b-4eb7-8f2a-d86f8e9ee5b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46e4b8ae0a3646839edee06fe671cf04'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:56.026805 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43e519c-2761-4e75-a1b7-25d50be590ff admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:56.032963 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313af3de-b336-4679-a772-d01d274c4041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:56.033628 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313af3de-b336-4679-a772-d01d274c4041 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:56.033915 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313af3de-b336-4679-a772-d01d274c4041 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:56.034184 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313af3de-b336-4679-a772-d01d274c4041 admin admin] PATH_INFO: `/v3/projects/89e0b5bdccf44702b0490185476f88b3/users/fd9a6286db4945d4a052c40a5242d409/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:56.045548 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1153/2289] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:18:55 2022] DELETE /identity/v3/users/46e4b8ae0a3646839edee06fe671cf04 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:18:56.049814 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe897ba-22cb-4113-8d29-ee88436d4c38 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:56.050078 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe897ba-22cb-4113-8d29-ee88436d4c38 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:56.050478 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe897ba-22cb-4113-8d29-ee88436d4c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:56.057224 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313af3de-b336-4679-a772-d01d274c4041 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89e0b5bdccf44702b0490185476f88b3, user_id=fd9a6286db4945d4a052c40a5242d409, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:56.059961 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313af3de-b336-4679-a772-d01d274c4041 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:56.075590 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1137/2290] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:18:56 2022] PUT /identity/v3/projects/89e0b5bdccf44702b0490185476f88b3/users/fd9a6286db4945d4a052c40a5242d409/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:18:56.077186 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-6fe897ba-22cb-4113-8d29-ee88436d4c38 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:18:56.078461 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6fe897ba-22cb-4113-8d29-ee88436d4c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:56.080711 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a926ce-1249-40d1-9cab-723f8d288788 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:56.081051 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a926ce-1249-40d1-9cab-723f8d288788 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:56.081350 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a926ce-1249-40d1-9cab-723f8d288788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:56.107977 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-60a926ce-1249-40d1-9cab-723f8d288788 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:18:56.109307 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-60a926ce-1249-40d1-9cab-723f8d288788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:56.115152 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:18:56.172900 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1154/2291] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:18:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:18:56.224055 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe897ba-22cb-4113-8d29-ee88436d4c38 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:56.224691 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1138/2292] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:18:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:18:56.226166 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6fe897ba-22cb-4113-8d29-ee88436d4c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:56.239003 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a926ce-1249-40d1-9cab-723f8d288788 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:56.242781 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-60a926ce-1249-40d1-9cab-723f8d288788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:56.343088 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe897ba-22cb-4113-8d29-ee88436d4c38 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:56.369022 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9565e9a7-fb09-4c73-a958-7e27a33a5015 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:56.369789 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9565e9a7-fb09-4c73-a958-7e27a33a5015 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:56.370072 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9565e9a7-fb09-4c73-a958-7e27a33a5015 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:56.370337 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9565e9a7-fb09-4c73-a958-7e27a33a5015 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:56.372511 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a926ce-1249-40d1-9cab-723f8d288788 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:56.377883 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7beac56-96f6-4d57-bdae-ec587b0ff30d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:56.378728 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7beac56-96f6-4d57-bdae-ec587b0ff30d service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:56.379095 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7beac56-96f6-4d57-bdae-ec587b0ff30d service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:56.379464 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7beac56-96f6-4d57-bdae-ec587b0ff30d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:56.392796 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7beac56-96f6-4d57-bdae-ec587b0ff30d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:56.395499 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7beac56-96f6-4d57-bdae-ec587b0ff30d service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:56.407155 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9565e9a7-fb09-4c73-a958-7e27a33a5015 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:56.408732 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9565e9a7-fb09-4c73-a958-7e27a33a5015 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:56.416979 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1139/2293] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:18:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:18:56.449868 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1155/2294] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:18:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:18:57.148352 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7beac56-96f6-4d57-bdae-ec587b0ff30d service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:57.149937 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a7beac56-96f6-4d57-bdae-ec587b0ff30d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:57.170693 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7beac56-96f6-4d57-bdae-ec587b0ff30d service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:57.179336 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b85312-b77e-45f8-8661-f6cad301c1b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:57.179922 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b85312-b77e-45f8-8661-f6cad301c1b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:57.180201 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b85312-b77e-45f8-8661-f6cad301c1b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:57.180540 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b85312-b77e-45f8-8661-f6cad301c1b8 admin admin] PATH_INFO: `/v3/projects/ad3bd3aa92c24f3f9324d0faf40e3292` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:57.182187 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b85312-b77e-45f8-8661-f6cad301c1b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad3bd3aa92c24f3f9324d0faf40e3292)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:57.187564 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b85312-b77e-45f8-8661-f6cad301c1b8 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:57.295765 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-53b85312-b77e-45f8-8661-f6cad301c1b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad3bd3aa92c24f3f9324d0faf40e3292', 'request_id': 'req-53b85312-b77e-45f8-8661-f6cad301c1b8'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:57.307793 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-53b85312-b77e-45f8-8661-f6cad301c1b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad3bd3aa92c24f3f9324d0faf40e3292', 'request_id': 'req-53b85312-b77e-45f8-8661-f6cad301c1b8'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:57.315668 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-53b85312-b77e-45f8-8661-f6cad301c1b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad3bd3aa92c24f3f9324d0faf40e3292', 'request_id': 'req-53b85312-b77e-45f8-8661-f6cad301c1b8'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:57.316839 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:18:57.333487 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-53b85312-b77e-45f8-8661-f6cad301c1b8 admin admin] The token cache is being invalidate because project ad3bd3aa92c24f3f9324d0faf40e3292 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:18:57.333780 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-53b85312-b77e-45f8-8661-f6cad301c1b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:57.334450 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-53b85312-b77e-45f8-8661-f6cad301c1b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:57.336877 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1140/2295] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:18:57 2022] DELETE /identity/v3/projects/ad3bd3aa92c24f3f9324d0faf40e3292 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:18:57.342747 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9565e9a7-fb09-4c73-a958-7e27a33a5015 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:57.354069 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9565e9a7-fb09-4c73-a958-7e27a33a5015 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:57.432264 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92eab46-0a43-42b1-96f8-d87a50858250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:57.433641 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92eab46-0a43-42b1-96f8-d87a50858250 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:57.433641 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92eab46-0a43-42b1-96f8-d87a50858250 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:57.434001 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92eab46-0a43-42b1-96f8-d87a50858250 admin admin] PATH_INFO: `/v3/users/d3b215a77642439e8d1fdb8a13bed2b5` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:57.435676 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92eab46-0a43-42b1-96f8-d87a50858250 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3b215a77642439e8d1fdb8a13bed2b5)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:57.439211 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92eab46-0a43-42b1-96f8-d87a50858250 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:57.520801 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a92eab46-0a43-42b1-96f8-d87a50858250 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3b215a77642439e8d1fdb8a13bed2b5'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:57.521254 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a92eab46-0a43-42b1-96f8-d87a50858250 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3b215a77642439e8d1fdb8a13bed2b5'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:57.549711 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a92eab46-0a43-42b1-96f8-d87a50858250 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3b215a77642439e8d1fdb8a13bed2b5'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:57.557764 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f3bd4a-ad07-41fa-b8b6-0d6c950df74c None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:57.558475 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f3bd4a-ad07-41fa-b8b6-0d6c950df74c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:57.558997 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f3bd4a-ad07-41fa-b8b6-0d6c950df74c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:57.567614 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a92eab46-0a43-42b1-96f8-d87a50858250 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3b215a77642439e8d1fdb8a13bed2b5'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:57.572260 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a92eab46-0a43-42b1-96f8-d87a50858250 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3b215a77642439e8d1fdb8a13bed2b5'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:57.578812 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:18:57.596517 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1156/2296] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:18:57 2022] DELETE /identity/v3/users/d3b215a77642439e8d1fdb8a13bed2b5 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:18:57.612613 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-71f3bd4a-ad07-41fa-b8b6-0d6c950df74c None None] MFA Rules not processed for user `fd9a6286db4945d4a052c40a5242d409`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:18:57.615521 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-71f3bd4a-ad07-41fa-b8b6-0d6c950df74c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:57.770313 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1141/2297] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:18:57 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:18:57.785368 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92eab46-0a43-42b1-96f8-d87a50858250 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:57.787061 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a92eab46-0a43-42b1-96f8-d87a50858250 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:57.910735 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92eab46-0a43-42b1-96f8-d87a50858250 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:57.914790 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066ccdaa-2234-498b-8cbe-b5f59a9f2e2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:57.915421 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ccdaa-2234-498b-8cbe-b5f59a9f2e2b service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:57.915698 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ccdaa-2234-498b-8cbe-b5f59a9f2e2b service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:57.915964 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ccdaa-2234-498b-8cbe-b5f59a9f2e2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:57.926023 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066ccdaa-2234-498b-8cbe-b5f59a9f2e2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:57.927667 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066ccdaa-2234-498b-8cbe-b5f59a9f2e2b service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:57.940241 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1157/2298] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:18:57 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:18:58.014906 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f3bd4a-ad07-41fa-b8b6-0d6c950df74c None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:58.018441 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-71f3bd4a-ad07-41fa-b8b6-0d6c950df74c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:58.107443 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f3bd4a-ad07-41fa-b8b6-0d6c950df74c None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:58.114147 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf05797f-5d9c-4907-87f6-56bcb2fdd81b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:58.115236 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf05797f-5d9c-4907-87f6-56bcb2fdd81b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:58.115505 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf05797f-5d9c-4907-87f6-56bcb2fdd81b admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:58.115777 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf05797f-5d9c-4907-87f6-56bcb2fdd81b admin admin] PATH_INFO: `/v3/projects/d4801e26a7a94835a7f170babc10946b` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:58.117005 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf05797f-5d9c-4907-87f6-56bcb2fdd81b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4801e26a7a94835a7f170babc10946b)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:58.119785 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf05797f-5d9c-4907-87f6-56bcb2fdd81b admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:58.179415 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-bf05797f-5d9c-4907-87f6-56bcb2fdd81b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4801e26a7a94835a7f170babc10946b', 'request_id': 'req-bf05797f-5d9c-4907-87f6-56bcb2fdd81b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:58.189016 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-bf05797f-5d9c-4907-87f6-56bcb2fdd81b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4801e26a7a94835a7f170babc10946b', 'request_id': 'req-bf05797f-5d9c-4907-87f6-56bcb2fdd81b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:58.195304 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-bf05797f-5d9c-4907-87f6-56bcb2fdd81b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4801e26a7a94835a7f170babc10946b', 'request_id': 'req-bf05797f-5d9c-4907-87f6-56bcb2fdd81b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:58.208876 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-bf05797f-5d9c-4907-87f6-56bcb2fdd81b admin admin] The token cache is being invalidate because project d4801e26a7a94835a7f170babc10946b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:18:58.209239 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-bf05797f-5d9c-4907-87f6-56bcb2fdd81b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:58.210404 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-bf05797f-5d9c-4907-87f6-56bcb2fdd81b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:58.212709 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1142/2299] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:18:58 2022] DELETE /identity/v3/projects/d4801e26a7a94835a7f170babc10946b => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:18:58.226364 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066ccdaa-2234-498b-8cbe-b5f59a9f2e2b service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:58.238521 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066ccdaa-2234-498b-8cbe-b5f59a9f2e2b service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:58.306543 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c70a7ee-9187-487d-8a32-88a11e932c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:58.307221 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c70a7ee-9187-487d-8a32-88a11e932c32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:58.307497 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c70a7ee-9187-487d-8a32-88a11e932c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:58.307737 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c70a7ee-9187-487d-8a32-88a11e932c32 admin admin] PATH_INFO: `/v3/users/5053aeade9c54c848fd6974babed6ba0` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:58.309015 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c70a7ee-9187-487d-8a32-88a11e932c32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5053aeade9c54c848fd6974babed6ba0)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:58.311359 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c70a7ee-9187-487d-8a32-88a11e932c32 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:58.363512 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9c70a7ee-9187-487d-8a32-88a11e932c32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5053aeade9c54c848fd6974babed6ba0'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:58.364362 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9c70a7ee-9187-487d-8a32-88a11e932c32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5053aeade9c54c848fd6974babed6ba0'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:58.373388 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9c70a7ee-9187-487d-8a32-88a11e932c32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5053aeade9c54c848fd6974babed6ba0'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:58.392983 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9c70a7ee-9187-487d-8a32-88a11e932c32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5053aeade9c54c848fd6974babed6ba0'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:58.394243 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9c70a7ee-9187-487d-8a32-88a11e932c32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5053aeade9c54c848fd6974babed6ba0'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:58.419344 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1158/2300] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:18:58 2022] DELETE /identity/v3/users/5053aeade9c54c848fd6974babed6ba0 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:18:58.744209 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf05797f-5d9c-4907-87f6-56bcb2fdd81b admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:58.745864 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-bf05797f-5d9c-4907-87f6-56bcb2fdd81b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:58.761091 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf05797f-5d9c-4907-87f6-56bcb2fdd81b admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:58.782856 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:18:58.840135 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7505259-3fff-4690-a86b-da5350409a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:58.840812 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7505259-3fff-4690-a86b-da5350409a36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:58.841102 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7505259-3fff-4690-a86b-da5350409a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:58.841369 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7505259-3fff-4690-a86b-da5350409a36 admin admin] PATH_INFO: `/v3/projects/77ac8fa62fad4945a3528f618d451a8e` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:58.842537 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7505259-3fff-4690-a86b-da5350409a36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77ac8fa62fad4945a3528f618d451a8e)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:58.844473 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7505259-3fff-4690-a86b-da5350409a36 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:58.908784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c7505259-3fff-4690-a86b-da5350409a36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77ac8fa62fad4945a3528f618d451a8e', 'request_id': 'req-c7505259-3fff-4690-a86b-da5350409a36'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:58.916708 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c7505259-3fff-4690-a86b-da5350409a36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77ac8fa62fad4945a3528f618d451a8e', 'request_id': 'req-c7505259-3fff-4690-a86b-da5350409a36'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:58.924334 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c7505259-3fff-4690-a86b-da5350409a36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77ac8fa62fad4945a3528f618d451a8e', 'request_id': 'req-c7505259-3fff-4690-a86b-da5350409a36'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:58.937831 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c7505259-3fff-4690-a86b-da5350409a36 admin admin] The token cache is being invalidate because project 77ac8fa62fad4945a3528f618d451a8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:18:58.938121 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c7505259-3fff-4690-a86b-da5350409a36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:58.938837 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c7505259-3fff-4690-a86b-da5350409a36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:58.940383 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1143/2301] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:18:58 2022] DELETE /identity/v3/projects/77ac8fa62fad4945a3528f618d451a8e => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:18:58.946434 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ebd714-aa73-448b-9e3b-ff5414d08446 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:58.946695 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ebd714-aa73-448b-9e3b-ff5414d08446 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:58.946950 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ebd714-aa73-448b-9e3b-ff5414d08446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:58.983010 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-c7ebd714-aa73-448b-9e3b-ff5414d08446 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:18:58.984275 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-c7ebd714-aa73-448b-9e3b-ff5414d08446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:59.000007 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:18:59.065051 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1159/2302] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:18:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:18:59.076150 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3402a-09a8-4478-8d79-f3c44fe2ad84 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:59.076425 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3402a-09a8-4478-8d79-f3c44fe2ad84 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:59.076701 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3402a-09a8-4478-8d79-f3c44fe2ad84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:59.097910 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-55e3402a-09a8-4478-8d79-f3c44fe2ad84 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:18:59.099427 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-55e3402a-09a8-4478-8d79-f3c44fe2ad84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:59.107839 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1144/2303] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:18:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:18:59.112633 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ebd714-aa73-448b-9e3b-ff5414d08446 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:59.124251 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ebd714-aa73-448b-9e3b-ff5414d08446 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:59.129520 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e3b089-2d8a-42da-8425-2a06967306a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:59.130122 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e3b089-2d8a-42da-8425-2a06967306a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:59.130399 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e3b089-2d8a-42da-8425-2a06967306a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:59.130663 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e3b089-2d8a-42da-8425-2a06967306a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:59.131315 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e3b089-2d8a-42da-8425-2a06967306a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:18:59.131607 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e3b089-2d8a-42da-8425-2a06967306a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:59.133293 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e3b089-2d8a-42da-8425-2a06967306a0 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:59.147637 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1160/2304] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 6 16:18:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:18:59.152520 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e3402a-09a8-4478-8d79-f3c44fe2ad84 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:59.162452 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e3402a-09a8-4478-8d79-f3c44fe2ad84 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:59.168250 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00c8bd4-c38e-43a3-a9ae-46805c5110de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:59.168907 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00c8bd4-c38e-43a3-a9ae-46805c5110de admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:59.169186 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00c8bd4-c38e-43a3-a9ae-46805c5110de admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:59.169446 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00c8bd4-c38e-43a3-a9ae-46805c5110de admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:59.170071 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00c8bd4-c38e-43a3-a9ae-46805c5110de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:59.171981 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00c8bd4-c38e-43a3-a9ae-46805c5110de admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:59.207279 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1145/2305] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:18:59 2022] POST /identity/v3/projects => generated 377 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:18:59.209492 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e3b089-2d8a-42da-8425-2a06967306a0 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:59.222290 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e3b089-2d8a-42da-8425-2a06967306a0 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:59.286374 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26e5878-ea42-4280-b539-8c1f1a758cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:59.287015 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e5878-ea42-4280-b539-8c1f1a758cef admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:59.287293 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e5878-ea42-4280-b539-8c1f1a758cef admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:59.287562 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e5878-ea42-4280-b539-8c1f1a758cef admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:59.288199 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26e5878-ea42-4280-b539-8c1f1a758cef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:59.290186 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26e5878-ea42-4280-b539-8c1f1a758cef admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:59.317062 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1161/2306] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:18:59 2022] POST /identity/v3/users => generated 289 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:18:59.318336 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00c8bd4-c38e-43a3-a9ae-46805c5110de admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:59.329990 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00c8bd4-c38e-43a3-a9ae-46805c5110de admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:59.335879 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef53ab68-89e9-47e2-a7e2-cbd5f2a66f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:59.336487 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef53ab68-89e9-47e2-a7e2-cbd5f2a66f18 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:59.336791 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef53ab68-89e9-47e2-a7e2-cbd5f2a66f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:59.337048 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef53ab68-89e9-47e2-a7e2-cbd5f2a66f18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:59.337691 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef53ab68-89e9-47e2-a7e2-cbd5f2a66f18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:59.339285 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef53ab68-89e9-47e2-a7e2-cbd5f2a66f18 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:59.350450 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1146/2307] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:18:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:18:59.369045 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26e5878-ea42-4280-b539-8c1f1a758cef admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:59.369045 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26e5878-ea42-4280-b539-8c1f1a758cef admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:59.371581 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdbea8a-4b5a-46f8-b06e-67c63d742724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:59.372240 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdbea8a-4b5a-46f8-b06e-67c63d742724 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:59.372515 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdbea8a-4b5a-46f8-b06e-67c63d742724 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:59.372816 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdbea8a-4b5a-46f8-b06e-67c63d742724 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:59.373487 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdbea8a-4b5a-46f8-b06e-67c63d742724 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:59.375183 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdbea8a-4b5a-46f8-b06e-67c63d742724 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:59.386998 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1162/2308] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:18:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:18:59.391620 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef53ab68-89e9-47e2-a7e2-cbd5f2a66f18 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:59.402482 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef53ab68-89e9-47e2-a7e2-cbd5f2a66f18 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:59.409847 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd794e2-1b07-4f75-abc6-1d15b08c5e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:59.410686 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd794e2-1b07-4f75-abc6-1d15b08c5e63 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:59.410973 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd794e2-1b07-4f75-abc6-1d15b08c5e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:59.411324 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd794e2-1b07-4f75-abc6-1d15b08c5e63 admin admin] PATH_INFO: `/v3/projects/6cef038328e64d08962ca22a7f37b86d/users/709dea8327934d27af036b9b3851414a/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:59.427686 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd794e2-1b07-4f75-abc6-1d15b08c5e63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cef038328e64d08962ca22a7f37b86d, user_id=709dea8327934d27af036b9b3851414a, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:59.430702 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd794e2-1b07-4f75-abc6-1d15b08c5e63 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:59.444980 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1147/2309] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:18:59 2022] PUT /identity/v3/projects/6cef038328e64d08962ca22a7f37b86d/users/709dea8327934d27af036b9b3851414a/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:18:59.457243 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1031cee2-4704-4f6b-a998-8bb716ff363b None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:59.457513 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1031cee2-4704-4f6b-a998-8bb716ff363b None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:59.457783 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1031cee2-4704-4f6b-a998-8bb716ff363b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:59.490103 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-1031cee2-4704-4f6b-a998-8bb716ff363b None None] MFA Rules not processed for user `709dea8327934d27af036b9b3851414a`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:18:59.491358 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1031cee2-4704-4f6b-a998-8bb716ff363b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:59.572451 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1163/2310] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:18:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:18:59.584691 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd794e2-1b07-4f75-abc6-1d15b08c5e63 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:59.586796 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-2fd794e2-1b07-4f75-abc6-1d15b08c5e63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:59.637539 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:18:59.654538 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd794e2-1b07-4f75-abc6-1d15b08c5e63 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:59.658483 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af62577-bcd0-42a1-930d-e8593066101c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:59.659089 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af62577-bcd0-42a1-930d-e8593066101c service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:59.659516 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af62577-bcd0-42a1-930d-e8593066101c service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:59.659801 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af62577-bcd0-42a1-930d-e8593066101c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:59.669970 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af62577-bcd0-42a1-930d-e8593066101c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:59.671467 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af62577-bcd0-42a1-930d-e8593066101c service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:59.680799 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1148/2311] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:18:59 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:19:00.506367 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64410e0f-abe4-4cb7-98d5-d43f04c988ef None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:19:00.506930 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64410e0f-abe4-4cb7-98d5-d43f04c988ef None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:19:00.506930 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64410e0f-abe4-4cb7-98d5-d43f04c988ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:19:00.528431 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-64410e0f-abe4-4cb7-98d5-d43f04c988ef None None] MFA Rules not processed for user `fd9a6286db4945d4a052c40a5242d409`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:19:00.529752 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-64410e0f-abe4-4cb7-98d5-d43f04c988ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:19:00.535616 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:19:00.591609 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1164/2312] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:19:00 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:19:09.676631 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=8 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:19:10.583702 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=20 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:19:11.381335 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d914326e-adc9-4e76-94f6-2430e9e41487 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:19:11.381809 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d914326e-adc9-4e76-94f6-2430e9e41487 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:19:11.381915 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d914326e-adc9-4e76-94f6-2430e9e41487 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:19:11.383437 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1149/2313] 10.222.0.43 () {54 vars in 961 bytes} [Tue Sep 6 16:19:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:19:11.388870 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371769fe-9681-40b8-b336-d86c5aa8bb56 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:19:11.389168 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371769fe-9681-40b8-b336-d86c5aa8bb56 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:19:11.389440 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371769fe-9681-40b8-b336-d86c5aa8bb56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:19:11.419257 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-371769fe-9681-40b8-b336-d86c5aa8bb56 None None] MFA Rules not processed for user `71c2912a44f04bd983b4d4112dbee734`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:19:11.420547 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-371769fe-9681-40b8-b336-d86c5aa8bb56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:19:11.516420 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1165/2314] 10.222.0.43 () {58 vars in 1058 bytes} [Tue Sep 6 16:19:11 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:19:11.524410 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d914326e-adc9-4e76-94f6-2430e9e41487 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:19:11.538393 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d914326e-adc9-4e76-94f6-2430e9e41487 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:19:11.543514 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f36d60-17cd-47f9-800b-a4b74e6ee9cc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71c2912a44f04bd983b4d4112dbee734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:19:11.544129 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f36d60-17cd-47f9-800b-a4b74e6ee9cc service placement] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:19:11.544405 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f36d60-17cd-47f9-800b-a4b74e6ee9cc service placement] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:19:11.544676 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f36d60-17cd-47f9-800b-a4b74e6ee9cc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:19:11.546483 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-75f36d60-17cd-47f9-800b-a4b74e6ee9cc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:19:11.558714 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f36d60-17cd-47f9-800b-a4b74e6ee9cc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:19:11.560678 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f36d60-17cd-47f9-800b-a4b74e6ee9cc service placement] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:19:11.571208 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1150/2315] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:19:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:19:21.502364 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=29 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:19:21.566819 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=6 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:19:26.177038 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371769fe-9681-40b8-b336-d86c5aa8bb56 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:19:26.290529 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371769fe-9681-40b8-b336-d86c5aa8bb56 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:19:26.296895 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7164d34-9769-4653-b847-c15e438edb71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:19:26.298892 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7164d34-9769-4653-b847-c15e438edb71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:19:26.299217 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7164d34-9769-4653-b847-c15e438edb71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:19:26.299528 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7164d34-9769-4653-b847-c15e438edb71 admin admin] PATH_INFO: `/v3/users/709dea8327934d27af036b9b3851414a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:19:26.300920 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7164d34-9769-4653-b847-c15e438edb71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=709dea8327934d27af036b9b3851414a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:19:26.303844 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7164d34-9769-4653-b847-c15e438edb71 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:19:26.444401 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d7164d34-9769-4653-b847-c15e438edb71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '709dea8327934d27af036b9b3851414a'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:19:26.445961 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d7164d34-9769-4653-b847-c15e438edb71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '709dea8327934d27af036b9b3851414a'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:19:26.456185 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d7164d34-9769-4653-b847-c15e438edb71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '709dea8327934d27af036b9b3851414a'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:19:26.482958 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d7164d34-9769-4653-b847-c15e438edb71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '709dea8327934d27af036b9b3851414a'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:19:26.483985 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d7164d34-9769-4653-b847-c15e438edb71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '709dea8327934d27af036b9b3851414a'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:19:26.504803 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1166/2316] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:19:26 2022] DELETE /identity/v3/users/709dea8327934d27af036b9b3851414a => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:19:26.509865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b655a97-26ff-4b5d-9c11-3810d452f9b5 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:19:26.510436 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b655a97-26ff-4b5d-9c11-3810d452f9b5 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:19:26.510526 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b655a97-26ff-4b5d-9c11-3810d452f9b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:19:26.548584 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-6b655a97-26ff-4b5d-9c11-3810d452f9b5 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:19:26.550212 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-6b655a97-26ff-4b5d-9c11-3810d452f9b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:19:26.634328 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1151/2317] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:19:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:19:26.644548 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7164d34-9769-4653-b847-c15e438edb71 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:19:26.646077 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d7164d34-9769-4653-b847-c15e438edb71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:19:26.717904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7164d34-9769-4653-b847-c15e438edb71 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:19:26.721896 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad7bd79-4482-4dac-a228-4b595a7b8192 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:19:26.722561 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad7bd79-4482-4dac-a228-4b595a7b8192 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:19:26.722836 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad7bd79-4482-4dac-a228-4b595a7b8192 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:19:26.723109 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad7bd79-4482-4dac-a228-4b595a7b8192 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:19:26.733302 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad7bd79-4482-4dac-a228-4b595a7b8192 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:19:26.734799 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad7bd79-4482-4dac-a228-4b595a7b8192 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:19:26.745316 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1167/2318] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:19:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:19:27.587914 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b655a97-26ff-4b5d-9c11-3810d452f9b5 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:19:27.589736 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-6b655a97-26ff-4b5d-9c11-3810d452f9b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:19:27.609314 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b655a97-26ff-4b5d-9c11-3810d452f9b5 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:19:27.616434 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec65bfc-a922-4f26-a12f-19cf2e93ee4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:19:27.617549 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec65bfc-a922-4f26-a12f-19cf2e93ee4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:19:27.618092 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec65bfc-a922-4f26-a12f-19cf2e93ee4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:19:27.618599 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec65bfc-a922-4f26-a12f-19cf2e93ee4d admin admin] PATH_INFO: `/v3/projects/6cef038328e64d08962ca22a7f37b86d` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:19:27.621239 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec65bfc-a922-4f26-a12f-19cf2e93ee4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cef038328e64d08962ca22a7f37b86d)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:19:27.623497 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec65bfc-a922-4f26-a12f-19cf2e93ee4d admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:19:27.717263 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-aec65bfc-a922-4f26-a12f-19cf2e93ee4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cef038328e64d08962ca22a7f37b86d', 'request_id': 'req-aec65bfc-a922-4f26-a12f-19cf2e93ee4d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:19:27.738339 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-aec65bfc-a922-4f26-a12f-19cf2e93ee4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cef038328e64d08962ca22a7f37b86d', 'request_id': 'req-aec65bfc-a922-4f26-a12f-19cf2e93ee4d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:19:27.761397 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-aec65bfc-a922-4f26-a12f-19cf2e93ee4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cef038328e64d08962ca22a7f37b86d', 'request_id': 'req-aec65bfc-a922-4f26-a12f-19cf2e93ee4d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:19:27.766871 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-aec65bfc-a922-4f26-a12f-19cf2e93ee4d admin admin] The token cache is being invalidate because project 6cef038328e64d08962ca22a7f37b86d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:19:27.767209 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-aec65bfc-a922-4f26-a12f-19cf2e93ee4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:19:27.767879 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-aec65bfc-a922-4f26-a12f-19cf2e93ee4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:19:27.770012 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1152/2319] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:19:27 2022] DELETE /identity/v3/projects/6cef038328e64d08962ca22a7f37b86d => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:19:28.089818 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad7bd79-4482-4dac-a228-4b595a7b8192 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:19:28.099357 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad7bd79-4482-4dac-a228-4b595a7b8192 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:19:28.126323 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:19:28.172601 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae409611-6b3f-4b07-9e74-a0578d7e53b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:19:28.173591 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae409611-6b3f-4b07-9e74-a0578d7e53b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:19:28.173591 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae409611-6b3f-4b07-9e74-a0578d7e53b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:19:28.173961 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae409611-6b3f-4b07-9e74-a0578d7e53b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:19:28.180193 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ae409611-6b3f-4b07-9e74-a0578d7e53b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:19:28.251497 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae409611-6b3f-4b07-9e74-a0578d7e53b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:19:28.253223 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae409611-6b3f-4b07-9e74-a0578d7e53b8 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:19:28.264718 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1168/2320] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:19:28 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:19:29.292981 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cdd1c7-45b3-4ba7-9862-47fd5642a0fb None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:19:29.293826 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cdd1c7-45b3-4ba7-9862-47fd5642a0fb None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:19:29.294593 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cdd1c7-45b3-4ba7-9862-47fd5642a0fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:19:29.319403 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-73cdd1c7-45b3-4ba7-9862-47fd5642a0fb None None] MFA Rules not processed for user `93096a58fcb34f2085a05ffa40df5937`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:19:29.320640 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-73cdd1c7-45b3-4ba7-9862-47fd5642a0fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:19:29.381522 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1153/2321] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:19:29 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:19:34.717125 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c002d90-601a-4590-b07d-618311da49f3 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:19:34.717675 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c002d90-601a-4590-b07d-618311da49f3 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:19:34.717675 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c002d90-601a-4590-b07d-618311da49f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:19:34.742518 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-4c002d90-601a-4590-b07d-618311da49f3 None None] MFA Rules not processed for user `d865006413b24ec3922619311f81f353`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:19:34.743739 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-4c002d90-601a-4590-b07d-618311da49f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:19:34.815181 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1169/2322] 10.222.0.43 () {58 vars in 1009 bytes} [Tue Sep 6 16:19:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:19:34.861148 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cdd1c7-45b3-4ba7-9862-47fd5642a0fb None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:19:34.863387 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-73cdd1c7-45b3-4ba7-9862-47fd5642a0fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:19:34.911611 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:19:34.967281 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cdd1c7-45b3-4ba7-9862-47fd5642a0fb None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:19:34.973714 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a979f1-c3a7-46c0-b317-fd652eb9ff8b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '312026c080b64aff9dd483d3591288a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:19:34.974346 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a979f1-c3a7-46c0-b317-fd652eb9ff8b service swift] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:19:34.974623 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a979f1-c3a7-46c0-b317-fd652eb9ff8b service swift] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:19:34.974920 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a979f1-c3a7-46c0-b317-fd652eb9ff8b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:19:34.985168 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a979f1-c3a7-46c0-b317-fd652eb9ff8b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:19:34.986802 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a979f1-c3a7-46c0-b317-fd652eb9ff8b service swift] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:19:34.996595 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1154/2323] 10.222.0.43 () {60 vars in 1384 bytes} [Tue Sep 6 16:19:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 16:19:44.804453 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=65 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:19:44.993277 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=19 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:19:55.563035 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897795c4-d032-4fda-8f89-bd52ca180eab None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:19:55.563622 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897795c4-d032-4fda-8f89-bd52ca180eab None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:19:55.563622 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897795c4-d032-4fda-8f89-bd52ca180eab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:19:55.585794 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-897795c4-d032-4fda-8f89-bd52ca180eab None None] MFA Rules not processed for user `d865006413b24ec3922619311f81f353`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:19:55.587244 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-897795c4-d032-4fda-8f89-bd52ca180eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:19:55.596389 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1170/2324] 10.222.0.43 () {58 vars in 967 bytes} [Tue Sep 6 16:19:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:19:55.609712 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a979f1-c3a7-46c0-b317-fd652eb9ff8b service swift] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:19:55.620551 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a979f1-c3a7-46c0-b317-fd652eb9ff8b service swift] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:19:55.626757 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249e3f41-d4f2-43b2-813a-20c4da4271a7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '312026c080b64aff9dd483d3591288a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:19:55.627354 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e3f41-d4f2-43b2-813a-20c4da4271a7 service swift] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:19:55.627624 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e3f41-d4f2-43b2-813a-20c4da4271a7 service swift] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:19:55.627942 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e3f41-d4f2-43b2-813a-20c4da4271a7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:19:55.636503 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249e3f41-d4f2-43b2-813a-20c4da4271a7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:19:55.638127 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249e3f41-d4f2-43b2-813a-20c4da4271a7 service swift] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:19:55.646938 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1155/2325] 10.222.0.43 () {60 vars in 1384 bytes} [Tue Sep 6 16:19:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 16:19:57.724959 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897795c4-d032-4fda-8f89-bd52ca180eab None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:19:57.726508 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-897795c4-d032-4fda-8f89-bd52ca180eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:19:57.805309 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897795c4-d032-4fda-8f89-bd52ca180eab None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:19:57.810796 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25557918-063b-49af-bd5a-e75b4a2eff04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:19:57.811479 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25557918-063b-49af-bd5a-e75b4a2eff04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:19:57.811759 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25557918-063b-49af-bd5a-e75b4a2eff04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:19:57.812031 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25557918-063b-49af-bd5a-e75b4a2eff04 admin admin] PATH_INFO: `/v3/users/2921386cc370487da26a786c8198c94c` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:19:57.813230 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25557918-063b-49af-bd5a-e75b4a2eff04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2921386cc370487da26a786c8198c94c)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:19:57.815408 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25557918-063b-49af-bd5a-e75b4a2eff04 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:19:57.869073 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-25557918-063b-49af-bd5a-e75b4a2eff04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2921386cc370487da26a786c8198c94c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:19:57.869690 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-25557918-063b-49af-bd5a-e75b4a2eff04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2921386cc370487da26a786c8198c94c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:19:57.876998 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-25557918-063b-49af-bd5a-e75b4a2eff04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2921386cc370487da26a786c8198c94c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:19:57.900575 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-25557918-063b-49af-bd5a-e75b4a2eff04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2921386cc370487da26a786c8198c94c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:19:57.901752 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-25557918-063b-49af-bd5a-e75b4a2eff04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2921386cc370487da26a786c8198c94c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:19:57.922617 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1171/2326] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:19:57 2022] DELETE /identity/v3/users/2921386cc370487da26a786c8198c94c => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:19:58.113456 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249e3f41-d4f2-43b2-813a-20c4da4271a7 service swift] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:19:58.125240 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249e3f41-d4f2-43b2-813a-20c4da4271a7 service swift] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:19:58.189263 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9c2143-5364-456f-b9e5-9aa9c44cd10f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:19:58.190706 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9c2143-5364-456f-b9e5-9aa9c44cd10f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:19:58.191092 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9c2143-5364-456f-b9e5-9aa9c44cd10f admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:19:58.191378 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9c2143-5364-456f-b9e5-9aa9c44cd10f admin admin] PATH_INFO: `/v3/projects/4a2dcd76a5f24bd7ba164032cbe22d20` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:19:58.192644 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9c2143-5364-456f-b9e5-9aa9c44cd10f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a2dcd76a5f24bd7ba164032cbe22d20)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:19:58.194611 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9c2143-5364-456f-b9e5-9aa9c44cd10f admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:19:58.269856 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-ed9c2143-5364-456f-b9e5-9aa9c44cd10f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a2dcd76a5f24bd7ba164032cbe22d20', 'request_id': 'req-ed9c2143-5364-456f-b9e5-9aa9c44cd10f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:19:58.294969 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-ed9c2143-5364-456f-b9e5-9aa9c44cd10f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a2dcd76a5f24bd7ba164032cbe22d20', 'request_id': 'req-ed9c2143-5364-456f-b9e5-9aa9c44cd10f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:19:58.304331 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-ed9c2143-5364-456f-b9e5-9aa9c44cd10f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a2dcd76a5f24bd7ba164032cbe22d20', 'request_id': 'req-ed9c2143-5364-456f-b9e5-9aa9c44cd10f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:19:58.320850 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-ed9c2143-5364-456f-b9e5-9aa9c44cd10f admin admin] The token cache is being invalidate because project 4a2dcd76a5f24bd7ba164032cbe22d20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:19:58.325254 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-ed9c2143-5364-456f-b9e5-9aa9c44cd10f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:19:58.327713 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-ed9c2143-5364-456f-b9e5-9aa9c44cd10f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:19:58.330762 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1156/2327] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:19:58 2022] DELETE /identity/v3/projects/4a2dcd76a5f24bd7ba164032cbe22d20 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:19:58.335586 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20856d45-e45a-4736-8ca1-3da1b2cd39d1 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:19:58.336080 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20856d45-e45a-4736-8ca1-3da1b2cd39d1 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:19:58.336187 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20856d45-e45a-4736-8ca1-3da1b2cd39d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:19:58.370214 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-20856d45-e45a-4736-8ca1-3da1b2cd39d1 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:19:58.371988 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-20856d45-e45a-4736-8ca1-3da1b2cd39d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:19:58.447600 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1172/2328] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:19:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:19:58.456258 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b83c35e-8532-4ac8-ae8e-b5048f2feca5 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:19:58.456542 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b83c35e-8532-4ac8-ae8e-b5048f2feca5 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:19:58.456855 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b83c35e-8532-4ac8-ae8e-b5048f2feca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:19:58.617504 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-8b83c35e-8532-4ac8-ae8e-b5048f2feca5 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:19:58.618733 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8b83c35e-8532-4ac8-ae8e-b5048f2feca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:19:58.627846 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1157/2329] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:19:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:19:58.632381 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20856d45-e45a-4736-8ca1-3da1b2cd39d1 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:19:58.646007 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20856d45-e45a-4736-8ca1-3da1b2cd39d1 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:19:58.651650 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e8058b-78e7-45be-a5fd-bf0f5757cdbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:19:58.652268 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e8058b-78e7-45be-a5fd-bf0f5757cdbb admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:19:58.652543 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e8058b-78e7-45be-a5fd-bf0f5757cdbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:19:58.652834 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e8058b-78e7-45be-a5fd-bf0f5757cdbb admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:19:58.653494 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e8058b-78e7-45be-a5fd-bf0f5757cdbb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:19:58.653794 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e8058b-78e7-45be-a5fd-bf0f5757cdbb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:19:58.655437 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e8058b-78e7-45be-a5fd-bf0f5757cdbb admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:19:58.667190 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:19:58.671573 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1173/2330] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 6 16:19:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:19:58.678070 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b83c35e-8532-4ac8-ae8e-b5048f2feca5 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:19:58.691592 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b83c35e-8532-4ac8-ae8e-b5048f2feca5 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:19:58.697409 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a5ef33-f83a-4915-990c-c00ba4129b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:19:58.698023 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a5ef33-f83a-4915-990c-c00ba4129b48 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:19:58.698317 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a5ef33-f83a-4915-990c-c00ba4129b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:19:58.698566 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a5ef33-f83a-4915-990c-c00ba4129b48 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:19:58.699210 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a5ef33-f83a-4915-990c-c00ba4129b48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:19:58.701187 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a5ef33-f83a-4915-990c-c00ba4129b48 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:19:58.735881 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1158/2331] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:19:58 2022] POST /identity/v3/projects => generated 365 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:19:58.742035 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e8058b-78e7-45be-a5fd-bf0f5757cdbb admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:19:58.752662 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e8058b-78e7-45be-a5fd-bf0f5757cdbb admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:19:58.818775 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8af966-8c39-4c2e-8161-7581e2629698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:19:58.819389 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8af966-8c39-4c2e-8161-7581e2629698 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:19:58.819661 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8af966-8c39-4c2e-8161-7581e2629698 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:19:58.819928 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8af966-8c39-4c2e-8161-7581e2629698 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:19:58.820565 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8af966-8c39-4c2e-8161-7581e2629698 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:19:58.822574 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8af966-8c39-4c2e-8161-7581e2629698 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:19:58.846855 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1174/2332] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:19:58 2022] POST /identity/v3/users => generated 283 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:19:58.851398 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a5ef33-f83a-4915-990c-c00ba4129b48 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:19:58.863134 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a5ef33-f83a-4915-990c-c00ba4129b48 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:19:58.868774 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1402277-0683-4ecd-8842-fb6a67b1023f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:19:58.869418 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1402277-0683-4ecd-8842-fb6a67b1023f admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:19:58.869688 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1402277-0683-4ecd-8842-fb6a67b1023f admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:19:58.869952 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1402277-0683-4ecd-8842-fb6a67b1023f admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:19:58.870625 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1402277-0683-4ecd-8842-fb6a67b1023f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:19:58.872290 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1402277-0683-4ecd-8842-fb6a67b1023f admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:19:58.883305 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1159/2333] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:19:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 16:19:58.887706 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8af966-8c39-4c2e-8161-7581e2629698 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:19:58.898561 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8af966-8c39-4c2e-8161-7581e2629698 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:19:58.905649 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ece39f-4262-47fb-8773-4ad644e6217f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:19:58.906324 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ece39f-4262-47fb-8773-4ad644e6217f admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:19:58.906574 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ece39f-4262-47fb-8773-4ad644e6217f admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:19:58.910369 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ece39f-4262-47fb-8773-4ad644e6217f admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:19:58.910369 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ece39f-4262-47fb-8773-4ad644e6217f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:19:58.910369 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ece39f-4262-47fb-8773-4ad644e6217f admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:19:58.921478 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1175/2334] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:19:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:19:58.926013 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1402277-0683-4ecd-8842-fb6a67b1023f admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:19:58.936982 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1402277-0683-4ecd-8842-fb6a67b1023f admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:19:58.942014 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b706c80b-b173-45b4-9c97-201aba402bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:19:58.942679 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b706c80b-b173-45b4-9c97-201aba402bcd admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:19:58.942975 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b706c80b-b173-45b4-9c97-201aba402bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:19:58.943228 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b706c80b-b173-45b4-9c97-201aba402bcd admin admin] PATH_INFO: `/v3/projects/9191469e88514ceda56db6485bfca6d9/users/02461521a46b416eaaadff254e04f3f9/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:19:58.958248 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b706c80b-b173-45b4-9c97-201aba402bcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=9191469e88514ceda56db6485bfca6d9, user_id=02461521a46b416eaaadff254e04f3f9, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:19:58.960933 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b706c80b-b173-45b4-9c97-201aba402bcd admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:19:58.974651 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1160/2335] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:19:58 2022] PUT /identity/v3/projects/9191469e88514ceda56db6485bfca6d9/users/02461521a46b416eaaadff254e04f3f9/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:19:58.985889 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8afbd-fffb-4ad4-b514-e04b6a29ca3a None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:19:58.986173 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8afbd-fffb-4ad4-b514-e04b6a29ca3a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:19:58.986440 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8afbd-fffb-4ad4-b514-e04b6a29ca3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:19:59.021818 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-d8e8afbd-fffb-4ad4-b514-e04b6a29ca3a None None] MFA Rules not processed for user `02461521a46b416eaaadff254e04f3f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:19:59.023057 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d8e8afbd-fffb-4ad4-b514-e04b6a29ca3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:19:59.093307 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1176/2336] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:19:58 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:19:59.105120 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b706c80b-b173-45b4-9c97-201aba402bcd admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:19:59.107124 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b706c80b-b173-45b4-9c97-201aba402bcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:19:59.122849 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:19:59.176673 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b706c80b-b173-45b4-9c97-201aba402bcd admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:19:59.179415 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf13172-bc45-458d-90f7-4ab779d66f30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:19:59.180017 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf13172-bc45-458d-90f7-4ab779d66f30 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:19:59.180285 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf13172-bc45-458d-90f7-4ab779d66f30 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:19:59.180547 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf13172-bc45-458d-90f7-4ab779d66f30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:19:59.190590 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf13172-bc45-458d-90f7-4ab779d66f30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:19:59.192019 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf13172-bc45-458d-90f7-4ab779d66f30 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:19:59.202005 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1161/2337] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:19:59 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:20:03.834097 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca910f-6741-4b67-a742-fde235ebaa68 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:20:03.834634 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca910f-6741-4b67-a742-fde235ebaa68 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:20:03.834634 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca910f-6741-4b67-a742-fde235ebaa68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:20:03.856687 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-b2ca910f-6741-4b67-a742-fde235ebaa68 None None] MFA Rules not processed for user `d865006413b24ec3922619311f81f353`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:20:03.857944 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b2ca910f-6741-4b67-a742-fde235ebaa68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:20:03.921030 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1177/2338] 10.222.0.43 () {58 vars in 1009 bytes} [Tue Sep 6 16:20:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:20:03.934196 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf13172-bc45-458d-90f7-4ab779d66f30 service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:20:03.936038 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-fcf13172-bc45-458d-90f7-4ab779d66f30 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:20:04.014775 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf13172-bc45-458d-90f7-4ab779d66f30 service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:20:04.019969 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81961c49-3f69-4911-9deb-081a2f22e183 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '312026c080b64aff9dd483d3591288a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:20:04.020595 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81961c49-3f69-4911-9deb-081a2f22e183 service swift] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:20:04.020896 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81961c49-3f69-4911-9deb-081a2f22e183 service swift] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:20:04.021166 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81961c49-3f69-4911-9deb-081a2f22e183 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:20:04.030634 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81961c49-3f69-4911-9deb-081a2f22e183 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:20:04.032221 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81961c49-3f69-4911-9deb-081a2f22e183 service swift] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:20:04.040936 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1162/2339] 10.222.0.43 () {60 vars in 1384 bytes} [Tue Sep 6 16:20:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 16:20:13.913182 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=91,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:20:14.037183 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=52 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:20:22.178833 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ddc657-103c-4b5d-bc3f-19d910806470 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:20:22.179471 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ddc657-103c-4b5d-bc3f-19d910806470 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:20:22.180049 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ddc657-103c-4b5d-bc3f-19d910806470 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:20:22.182892 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1178/2340] 10.222.0.43 () {54 vars in 921 bytes} [Tue Sep 6 16:20:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:20:22.189995 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a155e26d-d7a2-4d84-9cec-a19a6c1c43ce None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:20:22.190577 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a155e26d-d7a2-4d84-9cec-a19a6c1c43ce None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:20:22.190729 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a155e26d-d7a2-4d84-9cec-a19a6c1c43ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:20:22.219540 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-a155e26d-d7a2-4d84-9cec-a19a6c1c43ce None None] MFA Rules not processed for user `63de41f4348243998029f7d07aa5dc40`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:20:22.220844 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a155e26d-d7a2-4d84-9cec-a19a6c1c43ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:20:22.323795 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1163/2341] 10.222.0.43 () {58 vars in 1018 bytes} [Tue Sep 6 16:20:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:20:22.338086 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ddc657-103c-4b5d-bc3f-19d910806470 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:20:22.340111 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-02ddc657-103c-4b5d-bc3f-19d910806470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:20:22.405849 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ddc657-103c-4b5d-bc3f-19d910806470 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:20:22.409561 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbcc5cb-c3c4-44a4-980c-84071668f504 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c163b39e61648c385f065ce8091c3ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:20:22.410197 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbcc5cb-c3c4-44a4-980c-84071668f504 service glance] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:20:22.410471 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbcc5cb-c3c4-44a4-980c-84071668f504 service glance] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:20:22.410749 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbcc5cb-c3c4-44a4-980c-84071668f504 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:20:22.421799 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbcc5cb-c3c4-44a4-980c-84071668f504 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:20:22.423263 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbcc5cb-c3c4-44a4-980c-84071668f504 service glance] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:20:22.435508 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1179/2342] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:20:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:20:22.512486 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab508ca-6b7d-4442-884a-837ddd1ecc2e None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:20:22.512893 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab508ca-6b7d-4442-884a-837ddd1ecc2e None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:20:22.513151 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab508ca-6b7d-4442-884a-837ddd1ecc2e None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:20:22.514631 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1164/2343] 10.222.0.43 () {54 vars in 921 bytes} [Tue Sep 6 16:20:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:20:22.520183 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7730a78-983c-4bdb-8f22-65267eae49ed None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:20:22.520469 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7730a78-983c-4bdb-8f22-65267eae49ed None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:20:22.520786 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7730a78-983c-4bdb-8f22-65267eae49ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:20:22.550219 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-d7730a78-983c-4bdb-8f22-65267eae49ed None None] MFA Rules not processed for user `10565aefeaaa40549ed38bb9e0f4b189`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:20:22.551697 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d7730a78-983c-4bdb-8f22-65267eae49ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:20:22.629760 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1180/2344] 10.222.0.43 () {58 vars in 1018 bytes} [Tue Sep 6 16:20:22 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:20:22.642407 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab508ca-6b7d-4442-884a-837ddd1ecc2e None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:20:22.644846 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-aab508ca-6b7d-4442-884a-837ddd1ecc2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:20:22.661190 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab508ca-6b7d-4442-884a-837ddd1ecc2e None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:20:22.670445 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c4cd27-3ce0-40c0-9a47-40126b8874ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:20:22.671263 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c4cd27-3ce0-40c0-9a47-40126b8874ac service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:20:22.671541 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c4cd27-3ce0-40c0-9a47-40126b8874ac service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:20:22.671815 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c4cd27-3ce0-40c0-9a47-40126b8874ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:20:22.680249 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c4cd27-3ce0-40c0-9a47-40126b8874ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:20:22.681979 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c4cd27-3ce0-40c0-9a47-40126b8874ac service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:20:22.695226 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1165/2345] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:20:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:20:22.786583 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed170c65-dae6-41b6-9863-8e363fb22be6 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:20:22.786885 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed170c65-dae6-41b6-9863-8e363fb22be6 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:20:22.787133 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed170c65-dae6-41b6-9863-8e363fb22be6 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:20:22.788633 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1181/2346] 10.222.0.43 () {54 vars in 921 bytes} [Tue Sep 6 16:20:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:20:22.795765 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee35235-37ff-4183-a3ff-9e1bcd989e18 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:20:22.796281 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee35235-37ff-4183-a3ff-9e1bcd989e18 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:20:22.796708 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee35235-37ff-4183-a3ff-9e1bcd989e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:20:22.825125 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-9ee35235-37ff-4183-a3ff-9e1bcd989e18 None None] MFA Rules not processed for user `63de41f4348243998029f7d07aa5dc40`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:20:22.826214 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-9ee35235-37ff-4183-a3ff-9e1bcd989e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:20:22.836838 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1166/2347] 10.222.0.43 () {58 vars in 1018 bytes} [Tue Sep 6 16:20:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:20:22.847713 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed170c65-dae6-41b6-9863-8e363fb22be6 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:20:22.849648 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ed170c65-dae6-41b6-9863-8e363fb22be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:20:22.869028 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed170c65-dae6-41b6-9863-8e363fb22be6 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:20:22.872613 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6591cde0-183c-475b-b2b8-080d1fe1568e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:20:22.873259 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6591cde0-183c-475b-b2b8-080d1fe1568e service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:20:22.873533 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6591cde0-183c-475b-b2b8-080d1fe1568e service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:20:22.873802 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6591cde0-183c-475b-b2b8-080d1fe1568e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:20:22.906569 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6591cde0-183c-475b-b2b8-080d1fe1568e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:20:22.908049 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6591cde0-183c-475b-b2b8-080d1fe1568e service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:20:22.920944 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1182/2348] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:20:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:20:32.819900 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=40 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:20:32.915083 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=55 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:20:40.264097 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057c3218-3e4b-4174-a875-c80e516f40c6 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:20:40.264659 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057c3218-3e4b-4174-a875-c80e516f40c6 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:20:40.264659 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057c3218-3e4b-4174-a875-c80e516f40c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:20:40.287349 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-057c3218-3e4b-4174-a875-c80e516f40c6 None None] MFA Rules not processed for user `d865006413b24ec3922619311f81f353`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:20:40.290020 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-057c3218-3e4b-4174-a875-c80e516f40c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:20:40.296950 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1167/2349] 10.222.0.43 () {58 vars in 1009 bytes} [Tue Sep 6 16:20:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:20:40.323271 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6591cde0-183c-475b-b2b8-080d1fe1568e service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:20:40.334703 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6591cde0-183c-475b-b2b8-080d1fe1568e service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:20:40.340613 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068b79ab-2e07-4404-ac76-2a647a59f4f0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '312026c080b64aff9dd483d3591288a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:20:40.341469 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068b79ab-2e07-4404-ac76-2a647a59f4f0 service swift] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:20:40.341752 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068b79ab-2e07-4404-ac76-2a647a59f4f0 service swift] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:20:40.342017 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068b79ab-2e07-4404-ac76-2a647a59f4f0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:20:40.351312 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068b79ab-2e07-4404-ac76-2a647a59f4f0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:20:40.353013 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068b79ab-2e07-4404-ac76-2a647a59f4f0 service swift] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:20:40.371477 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1183/2350] 10.222.0.43 () {60 vars in 1384 bytes} [Tue Sep 6 16:20:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 16:20:46.773302 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057c3218-3e4b-4174-a875-c80e516f40c6 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:20:46.783330 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057c3218-3e4b-4174-a875-c80e516f40c6 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:20:46.787678 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ba5a2e-48f0-4e1f-878b-f124b98bf014 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:20:46.788291 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ba5a2e-48f0-4e1f-878b-f124b98bf014 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:20:46.788568 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ba5a2e-48f0-4e1f-878b-f124b98bf014 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:20:46.788941 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ba5a2e-48f0-4e1f-878b-f124b98bf014 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:20:46.790823 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-68ba5a2e-48f0-4e1f-878b-f124b98bf014 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:20:46.807748 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ba5a2e-48f0-4e1f-878b-f124b98bf014 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:20:46.809714 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ba5a2e-48f0-4e1f-878b-f124b98bf014 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:20:46.821167 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1168/2351] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:20:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:20:49.937247 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068b79ab-2e07-4404-ac76-2a647a59f4f0 service swift] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:20:49.950037 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068b79ab-2e07-4404-ac76-2a647a59f4f0 service swift] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:20:49.954757 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7936a5d8-a5c0-45ea-b149-d38bd94bb982 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:20:49.955435 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7936a5d8-a5c0-45ea-b149-d38bd94bb982 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:20:49.955816 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7936a5d8-a5c0-45ea-b149-d38bd94bb982 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:20:49.956094 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7936a5d8-a5c0-45ea-b149-d38bd94bb982 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:20:49.958830 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-7936a5d8-a5c0-45ea-b149-d38bd94bb982 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:20:49.982530 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7936a5d8-a5c0-45ea-b149-d38bd94bb982 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:20:49.985114 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7936a5d8-a5c0-45ea-b149-d38bd94bb982 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:20:50.004037 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1184/2352] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:20:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:20:55.191117 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ba5a2e-48f0-4e1f-878b-f124b98bf014 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:20:55.192977 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-68ba5a2e-48f0-4e1f-878b-f124b98bf014 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:20:55.207981 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ba5a2e-48f0-4e1f-878b-f124b98bf014 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:20:55.214757 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2271607-6db9-42b9-a186-f22c91a2d678 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63de41f4348243998029f7d07aa5dc40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:20:55.215376 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2271607-6db9-42b9-a186-f22c91a2d678 service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:20:55.215655 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2271607-6db9-42b9-a186-f22c91a2d678 service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:20:55.215928 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2271607-6db9-42b9-a186-f22c91a2d678 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:20:55.217853 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d2271607-6db9-42b9-a186-f22c91a2d678 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:20:55.290307 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2271607-6db9-42b9-a186-f22c91a2d678 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:20:55.291957 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2271607-6db9-42b9-a186-f22c91a2d678 service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:20:55.301768 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1169/2353] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:20:55 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:20:57.181799 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7936a5d8-a5c0-45ea-b149-d38bd94bb982 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:20:57.184082 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-7936a5d8-a5c0-45ea-b149-d38bd94bb982 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:20:57.198622 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7936a5d8-a5c0-45ea-b149-d38bd94bb982 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:20:57.205617 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25d8e41-aa55-407c-8536-c7fec8007fa5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63de41f4348243998029f7d07aa5dc40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:20:57.206229 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d8e41-aa55-407c-8536-c7fec8007fa5 service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:20:57.206507 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d8e41-aa55-407c-8536-c7fec8007fa5 service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:20:57.206779 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d8e41-aa55-407c-8536-c7fec8007fa5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:20:57.208938 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e25d8e41-aa55-407c-8536-c7fec8007fa5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:20:57.275738 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25d8e41-aa55-407c-8536-c7fec8007fa5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:20:57.277496 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25d8e41-aa55-407c-8536-c7fec8007fa5 service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:20:57.287746 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1185/2354] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:20:57 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:21:05.296992 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=37 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:21:07.282872 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=40 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:21:10.333927 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2271607-6db9-42b9-a186-f22c91a2d678 service nova] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:21:10.409214 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2271607-6db9-42b9-a186-f22c91a2d678 service nova] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:21:10.414420 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0102f3-3b84-499c-ab9a-a2586caea58a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:21:10.415093 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0102f3-3b84-499c-ab9a-a2586caea58a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:21:10.415385 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0102f3-3b84-499c-ab9a-a2586caea58a admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:21:10.415663 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0102f3-3b84-499c-ab9a-a2586caea58a admin admin] PATH_INFO: `/v3/users/02461521a46b416eaaadff254e04f3f9` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:21:10.417237 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0102f3-3b84-499c-ab9a-a2586caea58a admin admin] RBAC: Authorizing `identity:delete_user(user_id=02461521a46b416eaaadff254e04f3f9)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:21:10.419283 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0102f3-3b84-499c-ab9a-a2586caea58a admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:21:10.471566 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1a0102f3-3b84-499c-ab9a-a2586caea58a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02461521a46b416eaaadff254e04f3f9'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:21:10.472216 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1a0102f3-3b84-499c-ab9a-a2586caea58a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02461521a46b416eaaadff254e04f3f9'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:21:10.481493 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1a0102f3-3b84-499c-ab9a-a2586caea58a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02461521a46b416eaaadff254e04f3f9'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:21:10.498992 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1a0102f3-3b84-499c-ab9a-a2586caea58a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02461521a46b416eaaadff254e04f3f9'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:21:10.509853 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1a0102f3-3b84-499c-ab9a-a2586caea58a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02461521a46b416eaaadff254e04f3f9'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:21:10.517022 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1170/2355] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:21:10 2022] DELETE /identity/v3/users/02461521a46b416eaaadff254e04f3f9 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:21:10.522956 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac2390-15cc-4043-a584-36435dfddb18 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:21:10.523791 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac2390-15cc-4043-a584-36435dfddb18 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:21:10.523791 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac2390-15cc-4043-a584-36435dfddb18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:21:10.544183 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-15ac2390-15cc-4043-a584-36435dfddb18 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:21:10.545489 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-15ac2390-15cc-4043-a584-36435dfddb18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:21:10.632063 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1186/2356] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:21:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:21:10.642512 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0102f3-3b84-499c-ab9a-a2586caea58a admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:21:10.644034 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-1a0102f3-3b84-499c-ab9a-a2586caea58a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:21:10.711565 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0102f3-3b84-499c-ab9a-a2586caea58a admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:21:10.715604 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928b53a4-3888-451b-ba90-56613a1fd03f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:21:10.716401 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928b53a4-3888-451b-ba90-56613a1fd03f service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:21:10.716845 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928b53a4-3888-451b-ba90-56613a1fd03f service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:21:10.717182 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928b53a4-3888-451b-ba90-56613a1fd03f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:21:10.727157 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928b53a4-3888-451b-ba90-56613a1fd03f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:21:10.728597 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928b53a4-3888-451b-ba90-56613a1fd03f service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:21:10.739329 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1171/2357] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:21:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:21:11.088304 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ac2390-15cc-4043-a584-36435dfddb18 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:21:11.088304 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-15ac2390-15cc-4043-a584-36435dfddb18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:21:11.102610 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ac2390-15cc-4043-a584-36435dfddb18 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:21:11.111078 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7ced5e-1d6e-4a3b-a63d-37020174f773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:21:11.112264 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7ced5e-1d6e-4a3b-a63d-37020174f773 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:21:11.112792 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7ced5e-1d6e-4a3b-a63d-37020174f773 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:21:11.113245 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7ced5e-1d6e-4a3b-a63d-37020174f773 admin admin] PATH_INFO: `/v3/projects/9191469e88514ceda56db6485bfca6d9` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:21:11.114801 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7ced5e-1d6e-4a3b-a63d-37020174f773 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9191469e88514ceda56db6485bfca6d9)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:21:11.118366 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7ced5e-1d6e-4a3b-a63d-37020174f773 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:21:11.209941 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9b7ced5e-1d6e-4a3b-a63d-37020174f773 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9191469e88514ceda56db6485bfca6d9', 'request_id': 'req-9b7ced5e-1d6e-4a3b-a63d-37020174f773'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:21:11.217887 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9b7ced5e-1d6e-4a3b-a63d-37020174f773 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9191469e88514ceda56db6485bfca6d9', 'request_id': 'req-9b7ced5e-1d6e-4a3b-a63d-37020174f773'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:21:11.227111 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9b7ced5e-1d6e-4a3b-a63d-37020174f773 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9191469e88514ceda56db6485bfca6d9', 'request_id': 'req-9b7ced5e-1d6e-4a3b-a63d-37020174f773'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:21:11.242843 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9b7ced5e-1d6e-4a3b-a63d-37020174f773 admin admin] The token cache is being invalidate because project 9191469e88514ceda56db6485bfca6d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:21:11.243498 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9b7ced5e-1d6e-4a3b-a63d-37020174f773 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:21:11.244393 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9b7ced5e-1d6e-4a3b-a63d-37020174f773 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:21:11.246465 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1187/2358] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:21:11 2022] DELETE /identity/v3/projects/9191469e88514ceda56db6485bfca6d9 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:21:11.253861 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb1eef2-49d2-485e-947c-1b5e00ba3d5b None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:21:11.254358 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb1eef2-49d2-485e-947c-1b5e00ba3d5b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:21:11.254842 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb1eef2-49d2-485e-947c-1b5e00ba3d5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:21:11.282279 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-5eb1eef2-49d2-485e-947c-1b5e00ba3d5b None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:21:11.283504 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5eb1eef2-49d2-485e-947c-1b5e00ba3d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:21:11.305945 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:21:11.353252 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1172/2359] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:21:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:21:11.362716 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a71784-bd60-42d6-bb60-c4a034a385b7 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:21:11.362964 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a71784-bd60-42d6-bb60-c4a034a385b7 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:21:11.363259 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a71784-bd60-42d6-bb60-c4a034a385b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:21:11.383908 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-e3a71784-bd60-42d6-bb60-c4a034a385b7 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:21:11.385264 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e3a71784-bd60-42d6-bb60-c4a034a385b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:21:11.394753 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1188/2360] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:21:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:21:11.398261 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb1eef2-49d2-485e-947c-1b5e00ba3d5b None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:21:11.408635 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb1eef2-49d2-485e-947c-1b5e00ba3d5b None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:21:11.413463 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25b0773-0aa2-46a2-8bbd-4cc12a5ab24f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:21:11.414170 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b0773-0aa2-46a2-8bbd-4cc12a5ab24f admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:21:11.414442 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b0773-0aa2-46a2-8bbd-4cc12a5ab24f admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:21:11.414712 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b0773-0aa2-46a2-8bbd-4cc12a5ab24f admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:21:11.415378 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25b0773-0aa2-46a2-8bbd-4cc12a5ab24f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:21:11.415676 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25b0773-0aa2-46a2-8bbd-4cc12a5ab24f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:21:11.417333 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25b0773-0aa2-46a2-8bbd-4cc12a5ab24f admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:21:11.433477 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1173/2361] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 6 16:21:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:21:11.438121 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a71784-bd60-42d6-bb60-c4a034a385b7 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:21:11.448326 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a71784-bd60-42d6-bb60-c4a034a385b7 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:21:11.453695 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0a2879-4bf6-47b0-b322-a4d3eab58a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:21:11.454377 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0a2879-4bf6-47b0-b322-a4d3eab58a85 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:21:11.454592 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0a2879-4bf6-47b0-b322-a4d3eab58a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:21:11.454857 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0a2879-4bf6-47b0-b322-a4d3eab58a85 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:21:11.455513 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0a2879-4bf6-47b0-b322-a4d3eab58a85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:21:11.457496 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0a2879-4bf6-47b0-b322-a4d3eab58a85 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:21:11.488717 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1189/2362] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:21:11 2022] POST /identity/v3/projects => generated 369 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:21:11.492930 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25b0773-0aa2-46a2-8bbd-4cc12a5ab24f admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:21:11.503947 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25b0773-0aa2-46a2-8bbd-4cc12a5ab24f admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:21:11.580253 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdeedc5-2520-4e35-a8e4-0387f8a83d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:21:11.581007 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdeedc5-2520-4e35-a8e4-0387f8a83d06 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:21:11.581191 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdeedc5-2520-4e35-a8e4-0387f8a83d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:21:11.581575 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdeedc5-2520-4e35-a8e4-0387f8a83d06 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:21:11.582144 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdeedc5-2520-4e35-a8e4-0387f8a83d06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:21:11.584068 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdeedc5-2520-4e35-a8e4-0387f8a83d06 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:21:11.608808 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1174/2363] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:21:11 2022] POST /identity/v3/users => generated 285 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:21:11.611135 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0a2879-4bf6-47b0-b322-a4d3eab58a85 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:21:11.622708 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0a2879-4bf6-47b0-b322-a4d3eab58a85 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:21:11.628995 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286a7000-125c-4a3a-8766-8d2cf79f10d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:21:11.629753 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286a7000-125c-4a3a-8766-8d2cf79f10d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:21:11.630191 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286a7000-125c-4a3a-8766-8d2cf79f10d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:21:11.630366 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286a7000-125c-4a3a-8766-8d2cf79f10d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:21:11.630979 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286a7000-125c-4a3a-8766-8d2cf79f10d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:21:11.632905 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286a7000-125c-4a3a-8766-8d2cf79f10d7 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:21:11.646062 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1190/2364] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:21:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:21:11.651501 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdeedc5-2520-4e35-a8e4-0387f8a83d06 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:21:11.668424 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdeedc5-2520-4e35-a8e4-0387f8a83d06 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:21:11.676313 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b623e2b-eaa1-45b4-ab7e-f7b656b8bad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:21:11.677172 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b623e2b-eaa1-45b4-ab7e-f7b656b8bad3 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:21:11.677668 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b623e2b-eaa1-45b4-ab7e-f7b656b8bad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:21:11.678161 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b623e2b-eaa1-45b4-ab7e-f7b656b8bad3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:21:11.679323 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b623e2b-eaa1-45b4-ab7e-f7b656b8bad3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:21:11.681762 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b623e2b-eaa1-45b4-ab7e-f7b656b8bad3 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:21:11.694286 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1175/2365] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:21:11 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:21:11.698937 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286a7000-125c-4a3a-8766-8d2cf79f10d7 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:21:11.711702 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286a7000-125c-4a3a-8766-8d2cf79f10d7 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:21:11.717398 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa7b6cc-0e5d-4eb5-87c1-3804e3407b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:21:11.718105 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa7b6cc-0e5d-4eb5-87c1-3804e3407b88 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:21:11.718382 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa7b6cc-0e5d-4eb5-87c1-3804e3407b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:21:11.718656 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa7b6cc-0e5d-4eb5-87c1-3804e3407b88 admin admin] PATH_INFO: `/v3/projects/34c5034fbe9f4d2e9ee9c372dee6f2e6/users/42904081b7f74a3887e8ecb8e59b10c2/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:21:11.734536 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa7b6cc-0e5d-4eb5-87c1-3804e3407b88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34c5034fbe9f4d2e9ee9c372dee6f2e6, user_id=42904081b7f74a3887e8ecb8e59b10c2, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:21:11.737293 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa7b6cc-0e5d-4eb5-87c1-3804e3407b88 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:21:11.753297 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1191/2366] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:21:11 2022] PUT /identity/v3/projects/34c5034fbe9f4d2e9ee9c372dee6f2e6/users/42904081b7f74a3887e8ecb8e59b10c2/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:21:11.826042 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aa808e-93a7-42b0-b595-357ae162a551 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:21:11.826447 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aa808e-93a7-42b0-b595-357ae162a551 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:21:11.826880 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aa808e-93a7-42b0-b595-357ae162a551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:21:11.861709 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-c5aa808e-93a7-42b0-b595-357ae162a551 None None] MFA Rules not processed for user `42904081b7f74a3887e8ecb8e59b10c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:21:11.863073 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c5aa808e-93a7-42b0-b595-357ae162a551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:21:11.930362 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1176/2367] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:21:11 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:21:11.943577 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa7b6cc-0e5d-4eb5-87c1-3804e3407b88 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:21:11.945435 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3aa7b6cc-0e5d-4eb5-87c1-3804e3407b88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:21:11.972969 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:21:12.018087 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa7b6cc-0e5d-4eb5-87c1-3804e3407b88 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:21:12.021996 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec916b9f-7240-4350-985f-7ad1358d6c75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:21:12.022620 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec916b9f-7240-4350-985f-7ad1358d6c75 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:21:12.022984 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec916b9f-7240-4350-985f-7ad1358d6c75 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:21:12.023364 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec916b9f-7240-4350-985f-7ad1358d6c75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:21:12.035874 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec916b9f-7240-4350-985f-7ad1358d6c75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:21:12.037789 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec916b9f-7240-4350-985f-7ad1358d6c75 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:21:12.050179 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1192/2368] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:21:11 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:21:13.514208 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86398f63-d396-4c4a-b6ff-0341cd63b5c1 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:21:13.514774 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86398f63-d396-4c4a-b6ff-0341cd63b5c1 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:21:13.514774 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86398f63-d396-4c4a-b6ff-0341cd63b5c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:21:13.538339 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-86398f63-d396-4c4a-b6ff-0341cd63b5c1 None None] MFA Rules not processed for user `42904081b7f74a3887e8ecb8e59b10c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:21:13.540442 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-86398f63-d396-4c4a-b6ff-0341cd63b5c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:21:13.548739 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1177/2369] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:21:13 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:21:14.330340 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec916b9f-7240-4350-985f-7ad1358d6c75 service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:21:14.331800 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ec916b9f-7240-4350-985f-7ad1358d6c75 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:21:14.407642 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec916b9f-7240-4350-985f-7ad1358d6c75 service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:21:14.412655 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209a48d4-978b-42c6-b8e2-afa27ca96006 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71c2912a44f04bd983b4d4112dbee734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:21:14.413299 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209a48d4-978b-42c6-b8e2-afa27ca96006 service placement] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:21:14.413580 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209a48d4-978b-42c6-b8e2-afa27ca96006 service placement] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:21:14.413966 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209a48d4-978b-42c6-b8e2-afa27ca96006 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:21:14.415732 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-209a48d4-978b-42c6-b8e2-afa27ca96006 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:21:14.427438 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209a48d4-978b-42c6-b8e2-afa27ca96006 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:21:14.429129 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209a48d4-978b-42c6-b8e2-afa27ca96006 service placement] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:21:14.440524 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1193/2370] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:21:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:21:23.532468 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=92,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:21:24.435830 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=49 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:21:25.376413 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84e5518-f74d-4094-9691-111dadab64e5 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:21:25.377043 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84e5518-f74d-4094-9691-111dadab64e5 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:21:25.377043 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84e5518-f74d-4094-9691-111dadab64e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:21:25.396987 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-e84e5518-f74d-4094-9691-111dadab64e5 None None] MFA Rules not processed for user `fd9a6286db4945d4a052c40a5242d409`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:21:25.398221 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e84e5518-f74d-4094-9691-111dadab64e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:21:25.457962 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1178/2371] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:21:25 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:21:25.697048 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209a48d4-978b-42c6-b8e2-afa27ca96006 service placement] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:21:25.698654 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-209a48d4-978b-42c6-b8e2-afa27ca96006 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:21:25.757903 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209a48d4-978b-42c6-b8e2-afa27ca96006 service placement] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:21:25.762135 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614ece59-d2d9-46a8-9d34-99b1df12f1c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:21:25.762751 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ece59-d2d9-46a8-9d34-99b1df12f1c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:21:25.763027 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ece59-d2d9-46a8-9d34-99b1df12f1c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:21:25.763294 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ece59-d2d9-46a8-9d34-99b1df12f1c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:21:25.765305 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-614ece59-d2d9-46a8-9d34-99b1df12f1c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:21:25.775672 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614ece59-d2d9-46a8-9d34-99b1df12f1c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:21:25.777153 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614ece59-d2d9-46a8-9d34-99b1df12f1c5 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:21:25.785573 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1194/2372] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:21:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:21:26.046809 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e93577a-29f2-4c9f-ac42-14fbe8ace0f9 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:21:26.047098 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e93577a-29f2-4c9f-ac42-14fbe8ace0f9 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:21:26.047463 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e93577a-29f2-4c9f-ac42-14fbe8ace0f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:21:26.069223 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-8e93577a-29f2-4c9f-ac42-14fbe8ace0f9 None None] MFA Rules not processed for user `d865006413b24ec3922619311f81f353`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:21:26.070534 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8e93577a-29f2-4c9f-ac42-14fbe8ace0f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:21:26.150386 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1179/2373] 10.222.0.43 () {58 vars in 967 bytes} [Tue Sep 6 16:21:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:21:26.167392 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614ece59-d2d9-46a8-9d34-99b1df12f1c5 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:21:26.169009 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-614ece59-d2d9-46a8-9d34-99b1df12f1c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:21:26.291009 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614ece59-d2d9-46a8-9d34-99b1df12f1c5 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:21:26.303569 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29777e2f-326c-4f39-ad9d-b9a6ec25fc61 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '312026c080b64aff9dd483d3591288a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:21:26.306222 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29777e2f-326c-4f39-ad9d-b9a6ec25fc61 service swift] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:21:26.307117 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29777e2f-326c-4f39-ad9d-b9a6ec25fc61 service swift] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:21:26.308211 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29777e2f-326c-4f39-ad9d-b9a6ec25fc61 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:21:26.328646 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29777e2f-326c-4f39-ad9d-b9a6ec25fc61 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:21:26.334632 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29777e2f-326c-4f39-ad9d-b9a6ec25fc61 service swift] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:21:26.377148 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1195/2374] 10.222.0.43 () {60 vars in 1384 bytes} [Tue Sep 6 16:21:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 16:21:31.457130 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e93577a-29f2-4c9f-ac42-14fbe8ace0f9 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:21:31.467426 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e93577a-29f2-4c9f-ac42-14fbe8ace0f9 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:21:31.471285 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a1c15-c768-4621-894a-c19e54ef4b21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:21:31.471893 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a1c15-c768-4621-894a-c19e54ef4b21 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:21:31.472171 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a1c15-c768-4621-894a-c19e54ef4b21 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:21:31.472491 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a1c15-c768-4621-894a-c19e54ef4b21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:21:31.474793 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-656a1c15-c768-4621-894a-c19e54ef4b21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:21:31.485889 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656a1c15-c768-4621-894a-c19e54ef4b21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:21:31.487336 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656a1c15-c768-4621-894a-c19e54ef4b21 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:21:31.496478 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1180/2375] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:21:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:21:32.248029 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29777e2f-326c-4f39-ad9d-b9a6ec25fc61 service swift] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:21:32.250506 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-29777e2f-326c-4f39-ad9d-b9a6ec25fc61 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:21:32.332516 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29777e2f-326c-4f39-ad9d-b9a6ec25fc61 service swift] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:21:32.338853 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305620cc-8647-4644-aa21-b8d6848d20c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63de41f4348243998029f7d07aa5dc40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:21:32.339561 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305620cc-8647-4644-aa21-b8d6848d20c4 service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:21:32.339941 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305620cc-8647-4644-aa21-b8d6848d20c4 service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:21:32.340300 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305620cc-8647-4644-aa21-b8d6848d20c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:21:32.342320 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-305620cc-8647-4644-aa21-b8d6848d20c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:21:32.357680 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305620cc-8647-4644-aa21-b8d6848d20c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:21:32.359316 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305620cc-8647-4644-aa21-b8d6848d20c4 service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:21:32.371902 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1196/2376] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:21:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:21:41.492415 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=56 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:21:42.366584 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=92 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:21:45.598587 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8db223e-612f-49a7-9311-e825380a1c61 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:21:45.599096 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8db223e-612f-49a7-9311-e825380a1c61 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:21:45.599604 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8db223e-612f-49a7-9311-e825380a1c61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:21:45.632565 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-c8db223e-612f-49a7-9311-e825380a1c61 None None] MFA Rules not processed for user `d865006413b24ec3922619311f81f353`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:21:45.633989 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c8db223e-612f-49a7-9311-e825380a1c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:21:45.642498 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1181/2377] 10.222.0.43 () {58 vars in 1009 bytes} [Tue Sep 6 16:21:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:21:45.670325 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305620cc-8647-4644-aa21-b8d6848d20c4 service nova] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:21:45.684132 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305620cc-8647-4644-aa21-b8d6848d20c4 service nova] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:21:45.690809 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eff99c-0cd1-47a4-bf1c-62a856e5c46a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '312026c080b64aff9dd483d3591288a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:21:45.691442 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eff99c-0cd1-47a4-bf1c-62a856e5c46a service swift] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:21:45.691725 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eff99c-0cd1-47a4-bf1c-62a856e5c46a service swift] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:21:45.691999 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eff99c-0cd1-47a4-bf1c-62a856e5c46a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:21:45.700578 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eff99c-0cd1-47a4-bf1c-62a856e5c46a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:21:45.702265 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eff99c-0cd1-47a4-bf1c-62a856e5c46a service swift] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:21:45.710926 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1197/2378] 10.222.0.43 () {60 vars in 1384 bytes} [Tue Sep 6 16:21:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 16:21:46.290437 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab40ef2-4760-4f87-a32a-0b09c78c7cf9 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:21:46.290720 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab40ef2-4760-4f87-a32a-0b09c78c7cf9 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:21:46.290990 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab40ef2-4760-4f87-a32a-0b09c78c7cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:21:46.311361 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-bab40ef2-4760-4f87-a32a-0b09c78c7cf9 None None] MFA Rules not processed for user `d865006413b24ec3922619311f81f353`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:21:46.312545 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-bab40ef2-4760-4f87-a32a-0b09c78c7cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:21:46.322389 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1182/2379] 10.222.0.43 () {58 vars in 967 bytes} [Tue Sep 6 16:21:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:21:46.337331 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eff99c-0cd1-47a4-bf1c-62a856e5c46a service swift] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:21:46.348383 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eff99c-0cd1-47a4-bf1c-62a856e5c46a service swift] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:21:46.354264 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aace8b3c-3153-4f2e-bb32-d6a527668e39 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '312026c080b64aff9dd483d3591288a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:21:46.354861 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aace8b3c-3153-4f2e-bb32-d6a527668e39 service swift] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:21:46.355155 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aace8b3c-3153-4f2e-bb32-d6a527668e39 service swift] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:21:46.355369 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aace8b3c-3153-4f2e-bb32-d6a527668e39 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:21:46.377195 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aace8b3c-3153-4f2e-bb32-d6a527668e39 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:21:46.378853 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aace8b3c-3153-4f2e-bb32-d6a527668e39 service swift] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:21:46.387579 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1198/2380] 10.222.0.43 () {60 vars in 1384 bytes} [Tue Sep 6 16:21:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 16:21:46.629101 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41f218d-7d38-402c-a1b3-93d8389392af None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:21:46.629568 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41f218d-7d38-402c-a1b3-93d8389392af None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:21:46.629658 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41f218d-7d38-402c-a1b3-93d8389392af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:21:46.652534 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-b41f218d-7d38-402c-a1b3-93d8389392af None None] MFA Rules not processed for user `d865006413b24ec3922619311f81f353`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:21:46.653779 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b41f218d-7d38-402c-a1b3-93d8389392af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:21:46.661416 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1183/2381] 10.222.0.43 () {58 vars in 1009 bytes} [Tue Sep 6 16:21:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:21:46.673778 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aace8b3c-3153-4f2e-bb32-d6a527668e39 service swift] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:21:46.691133 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aace8b3c-3153-4f2e-bb32-d6a527668e39 service swift] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:21:46.697017 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71341d52-34e5-4a96-bb69-2f9f13dac1a2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '312026c080b64aff9dd483d3591288a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:21:46.697618 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71341d52-34e5-4a96-bb69-2f9f13dac1a2 service swift] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:21:46.697895 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71341d52-34e5-4a96-bb69-2f9f13dac1a2 service swift] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:21:46.698171 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71341d52-34e5-4a96-bb69-2f9f13dac1a2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:21:46.707040 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71341d52-34e5-4a96-bb69-2f9f13dac1a2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:21:46.708649 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71341d52-34e5-4a96-bb69-2f9f13dac1a2 service swift] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:21:46.716889 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1199/2382] 10.222.0.43 () {60 vars in 1384 bytes} [Tue Sep 6 16:21:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 16:21:47.078192 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd562a4-ce7a-4878-bb38-0e37ba6bc64e None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:21:47.078473 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd562a4-ce7a-4878-bb38-0e37ba6bc64e None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:21:47.078740 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd562a4-ce7a-4878-bb38-0e37ba6bc64e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:21:47.098781 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-abd562a4-ce7a-4878-bb38-0e37ba6bc64e None None] MFA Rules not processed for user `d865006413b24ec3922619311f81f353`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:21:47.099903 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-abd562a4-ce7a-4878-bb38-0e37ba6bc64e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:21:47.107576 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1184/2383] 10.222.0.43 () {58 vars in 967 bytes} [Tue Sep 6 16:21:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:21:47.120683 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71341d52-34e5-4a96-bb69-2f9f13dac1a2 service swift] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:21:47.131840 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71341d52-34e5-4a96-bb69-2f9f13dac1a2 service swift] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:21:47.137897 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e03691f-7b4f-4a05-b89b-ce7fd778bd3f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '312026c080b64aff9dd483d3591288a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:21:47.138511 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e03691f-7b4f-4a05-b89b-ce7fd778bd3f service swift] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:21:47.138789 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e03691f-7b4f-4a05-b89b-ce7fd778bd3f service swift] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:21:47.139061 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e03691f-7b4f-4a05-b89b-ce7fd778bd3f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:21:47.148134 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e03691f-7b4f-4a05-b89b-ce7fd778bd3f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:21:47.149780 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e03691f-7b4f-4a05-b89b-ce7fd778bd3f service swift] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:21:47.158516 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1200/2384] 10.222.0.43 () {60 vars in 1384 bytes} [Tue Sep 6 16:21:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 16:21:49.382140 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd562a4-ce7a-4878-bb38-0e37ba6bc64e None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:21:49.452311 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd562a4-ce7a-4878-bb38-0e37ba6bc64e None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:21:49.457422 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1dee76-7630-4e33-bebb-58f98253bdd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:21:49.458755 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1dee76-7630-4e33-bebb-58f98253bdd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:21:49.459009 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1dee76-7630-4e33-bebb-58f98253bdd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:21:49.459287 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1dee76-7630-4e33-bebb-58f98253bdd9 admin admin] PATH_INFO: `/v3/users/42904081b7f74a3887e8ecb8e59b10c2` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:21:49.460922 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1dee76-7630-4e33-bebb-58f98253bdd9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42904081b7f74a3887e8ecb8e59b10c2)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:21:49.462862 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1dee76-7630-4e33-bebb-58f98253bdd9 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:21:49.512909 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-dd1dee76-7630-4e33-bebb-58f98253bdd9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42904081b7f74a3887e8ecb8e59b10c2'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:21:49.513569 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-dd1dee76-7630-4e33-bebb-58f98253bdd9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42904081b7f74a3887e8ecb8e59b10c2'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:21:49.521257 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-dd1dee76-7630-4e33-bebb-58f98253bdd9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42904081b7f74a3887e8ecb8e59b10c2'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:21:49.533244 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-dd1dee76-7630-4e33-bebb-58f98253bdd9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42904081b7f74a3887e8ecb8e59b10c2'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:21:49.541766 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-dd1dee76-7630-4e33-bebb-58f98253bdd9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42904081b7f74a3887e8ecb8e59b10c2'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:21:49.548380 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1185/2385] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:21:49 2022] DELETE /identity/v3/users/42904081b7f74a3887e8ecb8e59b10c2 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:21:49.553185 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad673c3f-da9e-42ef-bb5b-950109afeee7 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:21:49.554990 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad673c3f-da9e-42ef-bb5b-950109afeee7 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:21:49.554990 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad673c3f-da9e-42ef-bb5b-950109afeee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:21:49.577119 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-ad673c3f-da9e-42ef-bb5b-950109afeee7 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:21:49.578578 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ad673c3f-da9e-42ef-bb5b-950109afeee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:21:49.646746 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1201/2386] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:21:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:21:49.657039 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1dee76-7630-4e33-bebb-58f98253bdd9 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:21:49.658695 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-dd1dee76-7630-4e33-bebb-58f98253bdd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:21:49.722640 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1dee76-7630-4e33-bebb-58f98253bdd9 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:21:49.726687 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b99b02f-14a1-420d-b685-1f0036971056 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:21:49.727310 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b99b02f-14a1-420d-b685-1f0036971056 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:21:49.727587 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b99b02f-14a1-420d-b685-1f0036971056 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:21:49.727857 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b99b02f-14a1-420d-b685-1f0036971056 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:21:49.735293 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:21:49.737744 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b99b02f-14a1-420d-b685-1f0036971056 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:21:49.739640 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b99b02f-14a1-420d-b685-1f0036971056 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:21:49.753631 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1186/2387] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:21:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:21:49.943755 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad673c3f-da9e-42ef-bb5b-950109afeee7 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:21:49.945302 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ad673c3f-da9e-42ef-bb5b-950109afeee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:21:49.960917 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad673c3f-da9e-42ef-bb5b-950109afeee7 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:21:49.965979 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f95a78b-e638-405c-8049-0cff95f77a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:21:49.966707 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f95a78b-e638-405c-8049-0cff95f77a3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:21:49.966965 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f95a78b-e638-405c-8049-0cff95f77a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:21:49.967256 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f95a78b-e638-405c-8049-0cff95f77a3a admin admin] PATH_INFO: `/v3/projects/34c5034fbe9f4d2e9ee9c372dee6f2e6` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:21:49.968478 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f95a78b-e638-405c-8049-0cff95f77a3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=34c5034fbe9f4d2e9ee9c372dee6f2e6)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:21:49.970496 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f95a78b-e638-405c-8049-0cff95f77a3a admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:21:50.019899 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-0f95a78b-e638-405c-8049-0cff95f77a3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34c5034fbe9f4d2e9ee9c372dee6f2e6', 'request_id': 'req-0f95a78b-e638-405c-8049-0cff95f77a3a'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:21:50.025724 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-0f95a78b-e638-405c-8049-0cff95f77a3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34c5034fbe9f4d2e9ee9c372dee6f2e6', 'request_id': 'req-0f95a78b-e638-405c-8049-0cff95f77a3a'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:21:50.034018 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-0f95a78b-e638-405c-8049-0cff95f77a3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34c5034fbe9f4d2e9ee9c372dee6f2e6', 'request_id': 'req-0f95a78b-e638-405c-8049-0cff95f77a3a'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:21:50.045919 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-0f95a78b-e638-405c-8049-0cff95f77a3a admin admin] The token cache is being invalidate because project 34c5034fbe9f4d2e9ee9c372dee6f2e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:21:50.046206 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-0f95a78b-e638-405c-8049-0cff95f77a3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:21:50.046859 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-0f95a78b-e638-405c-8049-0cff95f77a3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:21:50.048689 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1202/2388] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:21:49 2022] DELETE /identity/v3/projects/34c5034fbe9f4d2e9ee9c372dee6f2e6 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:21:50.053878 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db504e7-6ade-4901-adee-1fb33e9afef6 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:21:50.054168 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db504e7-6ade-4901-adee-1fb33e9afef6 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:21:50.054443 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db504e7-6ade-4901-adee-1fb33e9afef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:21:50.075641 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-3db504e7-6ade-4901-adee-1fb33e9afef6 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:21:50.076944 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-3db504e7-6ade-4901-adee-1fb33e9afef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:21:50.140774 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1187/2389] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:21:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:21:50.150666 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438a3512-a64a-46fd-94b5-37106ce6d324 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:21:50.151108 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438a3512-a64a-46fd-94b5-37106ce6d324 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:21:50.151421 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438a3512-a64a-46fd-94b5-37106ce6d324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:21:50.179501 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-438a3512-a64a-46fd-94b5-37106ce6d324 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:21:50.180712 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-438a3512-a64a-46fd-94b5-37106ce6d324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:21:50.189884 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1203/2390] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:21:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:21:50.194538 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db504e7-6ade-4901-adee-1fb33e9afef6 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:21:50.204842 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db504e7-6ade-4901-adee-1fb33e9afef6 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:21:50.210486 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ad274e-2c5c-4546-b51c-47816cea7ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:21:50.211094 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ad274e-2c5c-4546-b51c-47816cea7ec0 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:21:50.211396 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ad274e-2c5c-4546-b51c-47816cea7ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:21:50.211659 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ad274e-2c5c-4546-b51c-47816cea7ec0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:21:50.212304 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ad274e-2c5c-4546-b51c-47816cea7ec0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:21:50.212624 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ad274e-2c5c-4546-b51c-47816cea7ec0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:21:50.214235 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ad274e-2c5c-4546-b51c-47816cea7ec0 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:21:50.226709 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1188/2391] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 6 16:21:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:21:50.231368 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438a3512-a64a-46fd-94b5-37106ce6d324 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:21:50.241904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438a3512-a64a-46fd-94b5-37106ce6d324 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:21:50.247947 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b744e83a-de40-44c3-ae8f-63db23d3d24b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:21:50.248552 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b744e83a-de40-44c3-ae8f-63db23d3d24b admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:21:50.248857 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b744e83a-de40-44c3-ae8f-63db23d3d24b admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:21:50.249126 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b744e83a-de40-44c3-ae8f-63db23d3d24b admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:21:50.249802 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b744e83a-de40-44c3-ae8f-63db23d3d24b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:21:50.251821 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b744e83a-de40-44c3-ae8f-63db23d3d24b admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:21:50.286704 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1204/2392] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:21:50 2022] POST /identity/v3/projects => generated 379 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:21:50.291077 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ad274e-2c5c-4546-b51c-47816cea7ec0 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:21:50.301631 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ad274e-2c5c-4546-b51c-47816cea7ec0 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:21:50.372845 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ce4c23-18ec-466e-bcea-8439f1bb66a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:21:50.373457 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce4c23-18ec-466e-bcea-8439f1bb66a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:21:50.373729 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce4c23-18ec-466e-bcea-8439f1bb66a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:21:50.374003 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce4c23-18ec-466e-bcea-8439f1bb66a3 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:21:50.374626 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ce4c23-18ec-466e-bcea-8439f1bb66a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:21:50.376526 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ce4c23-18ec-466e-bcea-8439f1bb66a3 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:21:50.402188 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1189/2393] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:21:50 2022] POST /identity/v3/users => generated 290 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:21:50.407698 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b744e83a-de40-44c3-ae8f-63db23d3d24b admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:21:50.413476 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:21:50.424028 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b744e83a-de40-44c3-ae8f-63db23d3d24b admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:21:50.430989 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22827a17-da16-41f7-9bcf-a16c69ab768b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:21:50.431624 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22827a17-da16-41f7-9bcf-a16c69ab768b admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:21:50.432263 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22827a17-da16-41f7-9bcf-a16c69ab768b admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:21:50.432529 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22827a17-da16-41f7-9bcf-a16c69ab768b admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:21:50.433213 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22827a17-da16-41f7-9bcf-a16c69ab768b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:21:50.434829 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22827a17-da16-41f7-9bcf-a16c69ab768b admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:21:50.445298 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1205/2394] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:21:50 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 16:21:50.449466 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ce4c23-18ec-466e-bcea-8439f1bb66a3 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:21:50.459524 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ce4c23-18ec-466e-bcea-8439f1bb66a3 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:21:50.465022 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3917ef99-7a56-4730-8970-e97c917b592e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:21:50.465617 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3917ef99-7a56-4730-8970-e97c917b592e admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:21:50.465889 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3917ef99-7a56-4730-8970-e97c917b592e admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:21:50.466152 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3917ef99-7a56-4730-8970-e97c917b592e admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:21:50.466789 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3917ef99-7a56-4730-8970-e97c917b592e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:21:50.468430 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3917ef99-7a56-4730-8970-e97c917b592e admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:21:50.479952 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1190/2395] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:21:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:21:50.485785 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22827a17-da16-41f7-9bcf-a16c69ab768b admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:21:50.496271 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22827a17-da16-41f7-9bcf-a16c69ab768b admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:21:50.501188 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3228d40-5f9f-455f-9a9a-ecca35cfb628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:21:50.501853 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3228d40-5f9f-455f-9a9a-ecca35cfb628 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:21:50.502131 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3228d40-5f9f-455f-9a9a-ecca35cfb628 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:21:50.502403 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3228d40-5f9f-455f-9a9a-ecca35cfb628 admin admin] PATH_INFO: `/v3/projects/f90ec559752b44b2b67c765488814865/users/4d3856b7bb7f4f5bb801952b893bbe35/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:21:50.518607 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3228d40-5f9f-455f-9a9a-ecca35cfb628 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f90ec559752b44b2b67c765488814865, user_id=4d3856b7bb7f4f5bb801952b893bbe35, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:21:50.521308 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3228d40-5f9f-455f-9a9a-ecca35cfb628 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:21:50.538593 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1206/2396] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:21:50 2022] PUT /identity/v3/projects/f90ec559752b44b2b67c765488814865/users/4d3856b7bb7f4f5bb801952b893bbe35/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:21:50.543782 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a4559c-07df-4357-932d-c1eafe5090a6 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:21:50.544070 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a4559c-07df-4357-932d-c1eafe5090a6 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:21:50.544339 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a4559c-07df-4357-932d-c1eafe5090a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:21:50.564292 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-e5a4559c-07df-4357-932d-c1eafe5090a6 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:21:50.565480 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e5a4559c-07df-4357-932d-c1eafe5090a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:21:50.630175 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1191/2397] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:21:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:21:50.639900 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3228d40-5f9f-455f-9a9a-ecca35cfb628 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:21:50.641789 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a3228d40-5f9f-455f-9a9a-ecca35cfb628 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:21:50.698918 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3228d40-5f9f-455f-9a9a-ecca35cfb628 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:21:50.703310 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e991c3-cc29-47f4-a980-cfbd3d0a8932 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:21:50.703915 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e991c3-cc29-47f4-a980-cfbd3d0a8932 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:21:50.704187 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e991c3-cc29-47f4-a980-cfbd3d0a8932 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:21:50.704424 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e991c3-cc29-47f4-a980-cfbd3d0a8932 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:21:50.714285 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e991c3-cc29-47f4-a980-cfbd3d0a8932 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:21:50.715731 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e991c3-cc29-47f4-a980-cfbd3d0a8932 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:21:50.725169 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1207/2398] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:21:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:21:54.904508 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d25cb56-f573-4027-b5ec-7054e4dc44f9 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:21:54.904989 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d25cb56-f573-4027-b5ec-7054e4dc44f9 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:21:54.904989 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d25cb56-f573-4027-b5ec-7054e4dc44f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:21:54.912392 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:21:54.938749 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-4d25cb56-f573-4027-b5ec-7054e4dc44f9 None None] MFA Rules not processed for user `4d3856b7bb7f4f5bb801952b893bbe35`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:21:54.940310 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-4d25cb56-f573-4027-b5ec-7054e4dc44f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:21:55.026349 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1192/2399] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:21:54 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:21:55.031372 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e991c3-cc29-47f4-a980-cfbd3d0a8932 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:21:55.042688 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e991c3-cc29-47f4-a980-cfbd3d0a8932 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:21:55.055762 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18dd67b-2553-454e-8c45-aca1c2653452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:21:55.056451 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18dd67b-2553-454e-8c45-aca1c2653452 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:21:55.057425 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18dd67b-2553-454e-8c45-aca1c2653452 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:21:55.057713 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18dd67b-2553-454e-8c45-aca1c2653452 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:21:55.058544 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18dd67b-2553-454e-8c45-aca1c2653452 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:21:55.066055 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18dd67b-2553-454e-8c45-aca1c2653452 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:21:55.097614 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1208/2400] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:21:55 2022] POST /identity/v3/projects => generated 379 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:21:55.101349 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d25cb56-f573-4027-b5ec-7054e4dc44f9 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:21:55.119227 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d25cb56-f573-4027-b5ec-7054e4dc44f9 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:21:55.184090 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0056dcf6-f93b-4b9e-92bd-5baeeab057e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:21:55.184709 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0056dcf6-f93b-4b9e-92bd-5baeeab057e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:21:55.185009 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0056dcf6-f93b-4b9e-92bd-5baeeab057e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:21:55.185244 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0056dcf6-f93b-4b9e-92bd-5baeeab057e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:21:55.185877 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0056dcf6-f93b-4b9e-92bd-5baeeab057e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:21:55.187794 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0056dcf6-f93b-4b9e-92bd-5baeeab057e4 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:21:55.213073 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1193/2401] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:21:55 2022] POST /identity/v3/users => generated 296 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:21:55.217256 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18dd67b-2553-454e-8c45-aca1c2653452 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:21:55.228847 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18dd67b-2553-454e-8c45-aca1c2653452 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:21:55.237190 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b15b0e8-6150-4061-8a20-0b49fc381bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:21:55.237877 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b15b0e8-6150-4061-8a20-0b49fc381bc4 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:21:55.238161 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b15b0e8-6150-4061-8a20-0b49fc381bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:21:55.238431 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b15b0e8-6150-4061-8a20-0b49fc381bc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:21:55.239094 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b15b0e8-6150-4061-8a20-0b49fc381bc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:21:55.240806 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b15b0e8-6150-4061-8a20-0b49fc381bc4 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:21:55.278139 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1209/2402] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:21:55 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:21:55.279598 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0056dcf6-f93b-4b9e-92bd-5baeeab057e4 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:21:55.293114 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0056dcf6-f93b-4b9e-92bd-5baeeab057e4 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:21:55.300280 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fc1707-ccc1-4c91-8ff5-2b14a5b2c39b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:21:55.301007 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc1707-ccc1-4c91-8ff5-2b14a5b2c39b admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:21:55.301312 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc1707-ccc1-4c91-8ff5-2b14a5b2c39b admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:21:55.301630 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc1707-ccc1-4c91-8ff5-2b14a5b2c39b admin admin] PATH_INFO: `/v3/projects/8629993381d04142aea2ca6ae0a49690/users/002b7089d7424a73ad28612a6007c471/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:21:55.318430 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fc1707-ccc1-4c91-8ff5-2b14a5b2c39b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8629993381d04142aea2ca6ae0a49690, user_id=002b7089d7424a73ad28612a6007c471, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:21:55.321131 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fc1707-ccc1-4c91-8ff5-2b14a5b2c39b admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:21:55.335455 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1194/2403] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:21:55 2022] PUT /identity/v3/projects/8629993381d04142aea2ca6ae0a49690/users/002b7089d7424a73ad28612a6007c471/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:21:55.339555 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b15b0e8-6150-4061-8a20-0b49fc381bc4 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:21:55.350517 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b15b0e8-6150-4061-8a20-0b49fc381bc4 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:21:55.418207 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc299e5e-f38e-465a-b297-775238d28973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:21:55.418919 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc299e5e-f38e-465a-b297-775238d28973 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:21:55.419288 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc299e5e-f38e-465a-b297-775238d28973 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:21:55.419630 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc299e5e-f38e-465a-b297-775238d28973 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:21:55.420363 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc299e5e-f38e-465a-b297-775238d28973 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:21:55.422034 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc299e5e-f38e-465a-b297-775238d28973 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:21:55.434134 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1210/2404] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:21:55 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:21:55.438610 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fc1707-ccc1-4c91-8ff5-2b14a5b2c39b admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:21:55.453853 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fc1707-ccc1-4c91-8ff5-2b14a5b2c39b admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:21:55.459586 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4aaf9d-23b4-41c1-972b-f7dc6616e3e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:21:55.460287 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4aaf9d-23b4-41c1-972b-f7dc6616e3e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:21:55.460568 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4aaf9d-23b4-41c1-972b-f7dc6616e3e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:21:55.460861 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4aaf9d-23b4-41c1-972b-f7dc6616e3e6 admin admin] PATH_INFO: `/v3/projects/8629993381d04142aea2ca6ae0a49690/users/002b7089d7424a73ad28612a6007c471/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:21:55.462972 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4aaf9d-23b4-41c1-972b-f7dc6616e3e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8629993381d04142aea2ca6ae0a49690, user_id=002b7089d7424a73ad28612a6007c471, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:21:55.465687 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4aaf9d-23b4-41c1-972b-f7dc6616e3e6 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:21:55.479503 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1195/2405] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:21:55 2022] PUT /identity/v3/projects/8629993381d04142aea2ca6ae0a49690/users/002b7089d7424a73ad28612a6007c471/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:22:01.156783 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6932635-d3ff-4a1e-b5b2-2ea2e0dc3466 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:22:01.157353 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6932635-d3ff-4a1e-b5b2-2ea2e0dc3466 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:22:01.157444 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6932635-d3ff-4a1e-b5b2-2ea2e0dc3466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:22:01.190125 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-b6932635-d3ff-4a1e-b5b2-2ea2e0dc3466 None None] MFA Rules not processed for user `002b7089d7424a73ad28612a6007c471`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:22:01.191407 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b6932635-d3ff-4a1e-b5b2-2ea2e0dc3466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:22:01.213347 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:22:01.287902 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1211/2406] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:22:01 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:22:01.303008 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4aaf9d-23b4-41c1-972b-f7dc6616e3e6 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:22:01.304834 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7d4aaf9d-23b4-41c1-972b-f7dc6616e3e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:22:01.371858 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:22:01.401188 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4aaf9d-23b4-41c1-972b-f7dc6616e3e6 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:22:01.406969 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c945487-3220-48f9-8aa2-10fce1f2cc7f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63de41f4348243998029f7d07aa5dc40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:22:01.407596 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c945487-3220-48f9-8aa2-10fce1f2cc7f service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:22:01.407876 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c945487-3220-48f9-8aa2-10fce1f2cc7f service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:22:01.408146 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c945487-3220-48f9-8aa2-10fce1f2cc7f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:22:01.478375 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c945487-3220-48f9-8aa2-10fce1f2cc7f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:22:01.480033 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c945487-3220-48f9-8aa2-10fce1f2cc7f service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:22:01.489921 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1196/2407] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:22:01 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:22:11.284354 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=19 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:22:11.485373 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=36 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:22:34.128653 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934c8b2b-6bbf-4759-9e5c-a3c30b7dea61 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:22:34.129342 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934c8b2b-6bbf-4759-9e5c-a3c30b7dea61 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:22:34.129342 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934c8b2b-6bbf-4759-9e5c-a3c30b7dea61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:22:34.155858 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-934c8b2b-6bbf-4759-9e5c-a3c30b7dea61 None None] MFA Rules not processed for user `4d3856b7bb7f4f5bb801952b893bbe35`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:22:34.157214 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-934c8b2b-6bbf-4759-9e5c-a3c30b7dea61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:22:34.166462 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1212/2408] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:22:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:22:34.533661 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c945487-3220-48f9-8aa2-10fce1f2cc7f service nova] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:22:34.593067 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c945487-3220-48f9-8aa2-10fce1f2cc7f service nova] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:22:34.597018 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97e50cd-6e9e-4e2c-9ec2-c93ce4820ef4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:22:34.597633 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e50cd-6e9e-4e2c-9ec2-c93ce4820ef4 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:22:34.597907 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e50cd-6e9e-4e2c-9ec2-c93ce4820ef4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:22:34.598200 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e50cd-6e9e-4e2c-9ec2-c93ce4820ef4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:22:34.600302 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f97e50cd-6e9e-4e2c-9ec2-c93ce4820ef4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:22:34.611581 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97e50cd-6e9e-4e2c-9ec2-c93ce4820ef4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:22:34.613079 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97e50cd-6e9e-4e2c-9ec2-c93ce4820ef4 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:22:34.622347 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1197/2409] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:22:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:22:44.147986 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:22:44.618053 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=26 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:22:45.439991 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934c8b2b-6bbf-4759-9e5c-a3c30b7dea61 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:22:45.441630 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-934c8b2b-6bbf-4759-9e5c-a3c30b7dea61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:22:45.459878 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934c8b2b-6bbf-4759-9e5c-a3c30b7dea61 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:22:45.466930 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b0d635-65b3-45e4-85af-fff0fff9c9ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63de41f4348243998029f7d07aa5dc40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:22:45.467547 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b0d635-65b3-45e4-85af-fff0fff9c9ec service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:22:45.467815 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b0d635-65b3-45e4-85af-fff0fff9c9ec service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:22:45.468107 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b0d635-65b3-45e4-85af-fff0fff9c9ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:22:45.469998 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-95b0d635-65b3-45e4-85af-fff0fff9c9ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:22:45.537263 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b0d635-65b3-45e4-85af-fff0fff9c9ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:22:45.538904 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b0d635-65b3-45e4-85af-fff0fff9c9ec service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:22:45.548345 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1213/2410] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:22:45 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:22:48.085973 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97e50cd-6e9e-4e2c-9ec2-c93ce4820ef4 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:22:48.087626 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f97e50cd-6e9e-4e2c-9ec2-c93ce4820ef4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:22:48.170740 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97e50cd-6e9e-4e2c-9ec2-c93ce4820ef4 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:22:48.171228 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b0d635-65b3-45e4-85af-fff0fff9c9ec service nova] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:22:48.172938 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-95b0d635-65b3-45e4-85af-fff0fff9c9ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:22:48.178071 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbdf0f5-8b7c-4a8b-bdc5-342e12e2609f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71c2912a44f04bd983b4d4112dbee734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:22:48.179293 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbdf0f5-8b7c-4a8b-bdc5-342e12e2609f service placement] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:22:48.179434 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbdf0f5-8b7c-4a8b-bdc5-342e12e2609f service placement] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:22:48.179638 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbdf0f5-8b7c-4a8b-bdc5-342e12e2609f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:22:48.182556 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-abbdf0f5-8b7c-4a8b-bdc5-342e12e2609f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:22:48.197188 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbdf0f5-8b7c-4a8b-bdc5-342e12e2609f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:22:48.198778 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbdf0f5-8b7c-4a8b-bdc5-342e12e2609f service placement] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:22:48.209741 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1198/2411] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:22:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:22:48.265317 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b0d635-65b3-45e4-85af-fff0fff9c9ec service nova] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:22:48.269827 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6db7c09-5a29-4352-9f5b-0cbf24be736f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:22:48.270470 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6db7c09-5a29-4352-9f5b-0cbf24be736f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:22:48.270751 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6db7c09-5a29-4352-9f5b-0cbf24be736f admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:22:48.271026 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6db7c09-5a29-4352-9f5b-0cbf24be736f admin admin] PATH_INFO: `/v3/users/fd9a6286db4945d4a052c40a5242d409` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:22:48.272168 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6db7c09-5a29-4352-9f5b-0cbf24be736f admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd9a6286db4945d4a052c40a5242d409)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:22:48.274240 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6db7c09-5a29-4352-9f5b-0cbf24be736f admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:22:48.330303 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e6db7c09-5a29-4352-9f5b-0cbf24be736f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd9a6286db4945d4a052c40a5242d409'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:22:48.331000 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e6db7c09-5a29-4352-9f5b-0cbf24be736f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd9a6286db4945d4a052c40a5242d409'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:22:48.339845 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e6db7c09-5a29-4352-9f5b-0cbf24be736f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd9a6286db4945d4a052c40a5242d409'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:22:48.355007 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e6db7c09-5a29-4352-9f5b-0cbf24be736f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd9a6286db4945d4a052c40a5242d409'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:22:48.355810 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e6db7c09-5a29-4352-9f5b-0cbf24be736f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd9a6286db4945d4a052c40a5242d409'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:22:48.370621 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1214/2412] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:22:48 2022] DELETE /identity/v3/users/fd9a6286db4945d4a052c40a5242d409 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:22:48.570484 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbdf0f5-8b7c-4a8b-bdc5-342e12e2609f service placement] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:22:48.586598 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbdf0f5-8b7c-4a8b-bdc5-342e12e2609f service placement] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:22:48.653365 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ad72ef-827c-408c-97c4-3bc4469d0c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:22:48.654019 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ad72ef-827c-408c-97c4-3bc4469d0c72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:22:48.654294 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ad72ef-827c-408c-97c4-3bc4469d0c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:22:48.654587 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ad72ef-827c-408c-97c4-3bc4469d0c72 admin admin] PATH_INFO: `/v3/projects/89e0b5bdccf44702b0490185476f88b3` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:22:48.655879 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ad72ef-827c-408c-97c4-3bc4469d0c72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89e0b5bdccf44702b0490185476f88b3)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:22:48.657852 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ad72ef-827c-408c-97c4-3bc4469d0c72 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:22:48.707167 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d4ad72ef-827c-408c-97c4-3bc4469d0c72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89e0b5bdccf44702b0490185476f88b3', 'request_id': 'req-d4ad72ef-827c-408c-97c4-3bc4469d0c72'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:22:48.721571 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d4ad72ef-827c-408c-97c4-3bc4469d0c72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89e0b5bdccf44702b0490185476f88b3', 'request_id': 'req-d4ad72ef-827c-408c-97c4-3bc4469d0c72'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:22:48.728086 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d4ad72ef-827c-408c-97c4-3bc4469d0c72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89e0b5bdccf44702b0490185476f88b3', 'request_id': 'req-d4ad72ef-827c-408c-97c4-3bc4469d0c72'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:22:48.741136 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d4ad72ef-827c-408c-97c4-3bc4469d0c72 admin admin] The token cache is being invalidate because project 89e0b5bdccf44702b0490185476f88b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:22:48.741396 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d4ad72ef-827c-408c-97c4-3bc4469d0c72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:22:48.742079 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d4ad72ef-827c-408c-97c4-3bc4469d0c72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:22:48.743579 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1199/2413] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:22:48 2022] DELETE /identity/v3/projects/89e0b5bdccf44702b0490185476f88b3 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:22:51.426384 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6db7c09-5a29-4352-9f5b-0cbf24be736f admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:22:51.428659 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e6db7c09-5a29-4352-9f5b-0cbf24be736f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:22:51.452205 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:22:51.518124 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6db7c09-5a29-4352-9f5b-0cbf24be736f admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:22:51.591289 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2ca99e-e8aa-4791-808d-d21ecf039068 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63de41f4348243998029f7d07aa5dc40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:22:51.591936 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2ca99e-e8aa-4791-808d-d21ecf039068 service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:22:51.592210 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2ca99e-e8aa-4791-808d-d21ecf039068 service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:22:51.592594 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2ca99e-e8aa-4791-808d-d21ecf039068 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:22:51.594506 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ea2ca99e-e8aa-4791-808d-d21ecf039068 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:22:51.607589 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2ca99e-e8aa-4791-808d-d21ecf039068 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:22:51.609236 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2ca99e-e8aa-4791-808d-d21ecf039068 service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:22:51.623817 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1215/2414] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:22:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:22:55.449017 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ad72ef-827c-408c-97c4-3bc4469d0c72 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:22:55.450628 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d4ad72ef-827c-408c-97c4-3bc4469d0c72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:22:55.480090 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:22:55.549413 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ad72ef-827c-408c-97c4-3bc4469d0c72 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:22:55.553531 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb466e4-03e2-46f3-b2d4-6a9dee237260 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:22:55.554149 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb466e4-03e2-46f3-b2d4-6a9dee237260 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:22:55.554432 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb466e4-03e2-46f3-b2d4-6a9dee237260 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:22:55.554707 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb466e4-03e2-46f3-b2d4-6a9dee237260 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:22:55.557281 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7eb466e4-03e2-46f3-b2d4-6a9dee237260 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:22:55.630190 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb466e4-03e2-46f3-b2d4-6a9dee237260 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:22:55.631879 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb466e4-03e2-46f3-b2d4-6a9dee237260 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:22:55.643066 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1200/2415] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:22:55 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:23:01.616049 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=54 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:23:05.637910 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=41 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:23:14.195653 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8becad-5f6b-482a-9c1a-c90cd1fcb57c None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:23:14.196311 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8becad-5f6b-482a-9c1a-c90cd1fcb57c None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:23:14.196311 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8becad-5f6b-482a-9c1a-c90cd1fcb57c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:23:14.217307 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-9f8becad-5f6b-482a-9c1a-c90cd1fcb57c None None] MFA Rules not processed for user `93096a58fcb34f2085a05ffa40df5937`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:23:14.218540 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-9f8becad-5f6b-482a-9c1a-c90cd1fcb57c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:23:14.293693 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1216/2416] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:23:14 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:23:20.798076 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb466e4-03e2-46f3-b2d4-6a9dee237260 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:23:20.799733 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7eb466e4-03e2-46f3-b2d4-6a9dee237260 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:23:20.824075 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb466e4-03e2-46f3-b2d4-6a9dee237260 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:23:20.827356 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035c676c-bcfa-4034-bdc2-7a0098317fed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63de41f4348243998029f7d07aa5dc40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:23:20.827973 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035c676c-bcfa-4034-bdc2-7a0098317fed service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:23:20.828248 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035c676c-bcfa-4034-bdc2-7a0098317fed service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:23:20.828545 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035c676c-bcfa-4034-bdc2-7a0098317fed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:23:20.830774 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-035c676c-bcfa-4034-bdc2-7a0098317fed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:23:20.844727 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035c676c-bcfa-4034-bdc2-7a0098317fed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:23:20.846382 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035c676c-bcfa-4034-bdc2-7a0098317fed service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:23:20.860272 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1201/2417] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:23:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:23:24.274158 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=25 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:23:26.808448 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493854a1-74ba-4f1f-8f20-2854753307be None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:23:26.809159 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493854a1-74ba-4f1f-8f20-2854753307be None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:23:26.809159 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493854a1-74ba-4f1f-8f20-2854753307be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:23:26.830213 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-493854a1-74ba-4f1f-8f20-2854753307be None None] MFA Rules not processed for user `c3a65f26ab214feab60c7b6380539076`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:23:26.831537 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-493854a1-74ba-4f1f-8f20-2854753307be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:23:26.894074 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1217/2418] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:23:26 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:23:30.484743 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035c676c-bcfa-4034-bdc2-7a0098317fed service nova] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:23:30.496405 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035c676c-bcfa-4034-bdc2-7a0098317fed service nova] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:23:30.502356 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3337e1b3-7a35-4f01-bfc8-5f4abfa63629 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63de41f4348243998029f7d07aa5dc40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:23:30.502960 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3337e1b3-7a35-4f01-bfc8-5f4abfa63629 service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:23:30.503233 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3337e1b3-7a35-4f01-bfc8-5f4abfa63629 service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:23:30.503483 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3337e1b3-7a35-4f01-bfc8-5f4abfa63629 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:23:30.505341 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-3337e1b3-7a35-4f01-bfc8-5f4abfa63629 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:23:30.585657 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3337e1b3-7a35-4f01-bfc8-5f4abfa63629 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:23:30.587692 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3337e1b3-7a35-4f01-bfc8-5f4abfa63629 service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:23:30.599240 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1202/2419] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:23:30 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:23:36.884256 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=25 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:23:40.593783 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=34 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:23:42.622947 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf81c5-21c3-4f4d-ac13-f6959cd281c2 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:23:42.623497 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf81c5-21c3-4f4d-ac13-f6959cd281c2 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:23:42.623497 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf81c5-21c3-4f4d-ac13-f6959cd281c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:23:42.643318 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-e1cf81c5-21c3-4f4d-ac13-f6959cd281c2 None None] MFA Rules not processed for user `d865006413b24ec3922619311f81f353`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:23:42.644624 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e1cf81c5-21c3-4f4d-ac13-f6959cd281c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:23:42.712470 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1218/2420] 10.222.0.43 () {58 vars in 967 bytes} [Tue Sep 6 16:23:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:23:42.772718 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3337e1b3-7a35-4f01-bfc8-5f4abfa63629 service nova] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:23:42.774461 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-3337e1b3-7a35-4f01-bfc8-5f4abfa63629 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:23:42.862902 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3337e1b3-7a35-4f01-bfc8-5f4abfa63629 service nova] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:23:42.868379 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a332da-4b78-4017-a73b-edaa99ed8267 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '312026c080b64aff9dd483d3591288a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:23:42.869032 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a332da-4b78-4017-a73b-edaa99ed8267 service swift] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:23:42.869307 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a332da-4b78-4017-a73b-edaa99ed8267 service swift] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:23:42.869571 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a332da-4b78-4017-a73b-edaa99ed8267 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:23:42.878564 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a332da-4b78-4017-a73b-edaa99ed8267 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:23:42.880166 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a332da-4b78-4017-a73b-edaa99ed8267 service swift] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:23:42.895724 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1203/2421] 10.222.0.43 () {60 vars in 1384 bytes} [Tue Sep 6 16:23:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 16:23:43.303129 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cf81c5-21c3-4f4d-ac13-f6959cd281c2 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:23:43.305183 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e1cf81c5-21c3-4f4d-ac13-f6959cd281c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:23:43.319233 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cf81c5-21c3-4f4d-ac13-f6959cd281c2 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:23:43.322824 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c31ba3-c971-4683-a0e1-981dd2bfa67d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:23:43.323436 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c31ba3-c971-4683-a0e1-981dd2bfa67d service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:23:43.323710 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c31ba3-c971-4683-a0e1-981dd2bfa67d service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:23:43.324042 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c31ba3-c971-4683-a0e1-981dd2bfa67d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:23:43.325877 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-99c31ba3-c971-4683-a0e1-981dd2bfa67d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:23:43.392616 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c31ba3-c971-4683-a0e1-981dd2bfa67d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:23:43.394119 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c31ba3-c971-4683-a0e1-981dd2bfa67d service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:23:43.405410 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1219/2422] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:23:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:23:49.930277 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a332da-4b78-4017-a73b-edaa99ed8267 service swift] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:23:49.932223 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-87a332da-4b78-4017-a73b-edaa99ed8267 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:23:49.988989 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c31ba3-c971-4683-a0e1-981dd2bfa67d service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:23:49.990818 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-99c31ba3-c971-4683-a0e1-981dd2bfa67d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:23:50.079505 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c31ba3-c971-4683-a0e1-981dd2bfa67d service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:23:50.080117 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a332da-4b78-4017-a73b-edaa99ed8267 service swift] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:23:50.085404 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087fe442-6ab3-4763-bdc1-d327e428be2d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71c2912a44f04bd983b4d4112dbee734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:23:50.086088 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087fe442-6ab3-4763-bdc1-d327e428be2d service placement] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:23:50.086604 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087fe442-6ab3-4763-bdc1-d327e428be2d service placement] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:23:50.086852 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4950828a-2c25-4d86-830f-65845ae9b6f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:23:50.086987 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087fe442-6ab3-4763-bdc1-d327e428be2d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:23:50.087791 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4950828a-2c25-4d86-830f-65845ae9b6f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:23:50.088185 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4950828a-2c25-4d86-830f-65845ae9b6f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:23:50.088570 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4950828a-2c25-4d86-830f-65845ae9b6f1 admin admin] PATH_INFO: `/v3/users/bbe4c1ecf08f4dd5a005f883c16c6a62` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:23:50.088684 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-087fe442-6ab3-4763-bdc1-d327e428be2d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:23:50.090225 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4950828a-2c25-4d86-830f-65845ae9b6f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbe4c1ecf08f4dd5a005f883c16c6a62)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:23:50.093782 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4950828a-2c25-4d86-830f-65845ae9b6f1 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:23:50.100538 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087fe442-6ab3-4763-bdc1-d327e428be2d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:23:50.102218 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087fe442-6ab3-4763-bdc1-d327e428be2d service placement] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:23:50.112805 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1204/2423] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:23:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:23:50.143628 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:23:50.158354 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-4950828a-2c25-4d86-830f-65845ae9b6f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbe4c1ecf08f4dd5a005f883c16c6a62'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:23:50.159010 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-4950828a-2c25-4d86-830f-65845ae9b6f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbe4c1ecf08f4dd5a005f883c16c6a62'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:23:50.165952 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-4950828a-2c25-4d86-830f-65845ae9b6f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbe4c1ecf08f4dd5a005f883c16c6a62'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:23:50.178887 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-4950828a-2c25-4d86-830f-65845ae9b6f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbe4c1ecf08f4dd5a005f883c16c6a62'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:23:50.179986 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-4950828a-2c25-4d86-830f-65845ae9b6f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbe4c1ecf08f4dd5a005f883c16c6a62'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:23:50.203063 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1220/2424] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:23:49 2022] DELETE /identity/v3/users/bbe4c1ecf08f4dd5a005f883c16c6a62 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:23:50.391328 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087fe442-6ab3-4763-bdc1-d327e428be2d service placement] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:23:50.410186 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087fe442-6ab3-4763-bdc1-d327e428be2d service placement] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:23:50.449257 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:23:50.479504 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceacb433-3ada-472e-b221-169100040e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:23:50.480141 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceacb433-3ada-472e-b221-169100040e1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:23:50.480417 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceacb433-3ada-472e-b221-169100040e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:23:50.480689 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceacb433-3ada-472e-b221-169100040e1d admin admin] PATH_INFO: `/v3/projects/04bbe101d7ac40ab995da2159f3105cd` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:23:50.481878 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceacb433-3ada-472e-b221-169100040e1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=04bbe101d7ac40ab995da2159f3105cd)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:23:50.483788 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceacb433-3ada-472e-b221-169100040e1d admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:23:50.546320 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-ceacb433-3ada-472e-b221-169100040e1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04bbe101d7ac40ab995da2159f3105cd', 'request_id': 'req-ceacb433-3ada-472e-b221-169100040e1d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:23:50.555335 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-ceacb433-3ada-472e-b221-169100040e1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04bbe101d7ac40ab995da2159f3105cd', 'request_id': 'req-ceacb433-3ada-472e-b221-169100040e1d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:23:50.561026 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-ceacb433-3ada-472e-b221-169100040e1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04bbe101d7ac40ab995da2159f3105cd', 'request_id': 'req-ceacb433-3ada-472e-b221-169100040e1d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:23:50.572687 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-ceacb433-3ada-472e-b221-169100040e1d admin admin] The token cache is being invalidate because project 04bbe101d7ac40ab995da2159f3105cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:23:50.573008 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-ceacb433-3ada-472e-b221-169100040e1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:23:50.573698 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-ceacb433-3ada-472e-b221-169100040e1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:23:50.575288 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1205/2425] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:23:50 2022] DELETE /identity/v3/projects/04bbe101d7ac40ab995da2159f3105cd => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:23:50.579924 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4950828a-2c25-4d86-830f-65845ae9b6f1 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:23:50.581885 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-4950828a-2c25-4d86-830f-65845ae9b6f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:23:50.601179 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4950828a-2c25-4d86-830f-65845ae9b6f1 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:23:50.662960 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ad9f58-d3e4-4a41-9b2d-bfe720c8d77b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:23:50.663621 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad9f58-d3e4-4a41-9b2d-bfe720c8d77b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:23:50.663907 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad9f58-d3e4-4a41-9b2d-bfe720c8d77b admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:23:50.664203 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad9f58-d3e4-4a41-9b2d-bfe720c8d77b admin admin] PATH_INFO: `/v3/users/a280595c6ff4495ab9f4120b4bcc60c7` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:23:50.665392 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ad9f58-d3e4-4a41-9b2d-bfe720c8d77b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a280595c6ff4495ab9f4120b4bcc60c7)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:23:50.667329 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ad9f58-d3e4-4a41-9b2d-bfe720c8d77b admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:23:50.718225 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-78ad9f58-d3e4-4a41-9b2d-bfe720c8d77b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a280595c6ff4495ab9f4120b4bcc60c7'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:23:50.719103 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-78ad9f58-d3e4-4a41-9b2d-bfe720c8d77b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a280595c6ff4495ab9f4120b4bcc60c7'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:23:50.727367 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-78ad9f58-d3e4-4a41-9b2d-bfe720c8d77b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a280595c6ff4495ab9f4120b4bcc60c7'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:23:50.740271 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-78ad9f58-d3e4-4a41-9b2d-bfe720c8d77b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a280595c6ff4495ab9f4120b4bcc60c7'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:23:50.741061 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-78ad9f58-d3e4-4a41-9b2d-bfe720c8d77b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a280595c6ff4495ab9f4120b4bcc60c7'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:23:50.755441 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1221/2426] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:23:50 2022] DELETE /identity/v3/users/a280595c6ff4495ab9f4120b4bcc60c7 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:23:51.186356 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceacb433-3ada-472e-b221-169100040e1d admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:23:51.188098 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ceacb433-3ada-472e-b221-169100040e1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:23:51.205484 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceacb433-3ada-472e-b221-169100040e1d admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:23:51.268118 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2980a7b-4eda-4bc8-bae3-e75bde065310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:23:51.268780 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2980a7b-4eda-4bc8-bae3-e75bde065310 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:23:51.269081 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2980a7b-4eda-4bc8-bae3-e75bde065310 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:23:51.269360 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2980a7b-4eda-4bc8-bae3-e75bde065310 admin admin] PATH_INFO: `/v3/projects/009f1033f43945cd8483afafd38f32b3` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:23:51.270566 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2980a7b-4eda-4bc8-bae3-e75bde065310 admin admin] RBAC: Authorizing `identity:delete_project(project_id=009f1033f43945cd8483afafd38f32b3)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:23:51.272500 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2980a7b-4eda-4bc8-bae3-e75bde065310 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:23:51.330372 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c2980a7b-4eda-4bc8-bae3-e75bde065310 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '009f1033f43945cd8483afafd38f32b3', 'request_id': 'req-c2980a7b-4eda-4bc8-bae3-e75bde065310'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:23:51.341170 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c2980a7b-4eda-4bc8-bae3-e75bde065310 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '009f1033f43945cd8483afafd38f32b3', 'request_id': 'req-c2980a7b-4eda-4bc8-bae3-e75bde065310'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:23:51.351436 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:23:51.354269 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c2980a7b-4eda-4bc8-bae3-e75bde065310 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '009f1033f43945cd8483afafd38f32b3', 'request_id': 'req-c2980a7b-4eda-4bc8-bae3-e75bde065310'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:23:51.377786 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c2980a7b-4eda-4bc8-bae3-e75bde065310 admin admin] The token cache is being invalidate because project 009f1033f43945cd8483afafd38f32b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:23:51.378046 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c2980a7b-4eda-4bc8-bae3-e75bde065310 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:23:51.378804 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c2980a7b-4eda-4bc8-bae3-e75bde065310 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:23:51.380821 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1206/2427] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:23:51 2022] DELETE /identity/v3/projects/009f1033f43945cd8483afafd38f32b3 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:23:51.386439 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b792f1-cc79-415f-b5ae-b723824bc0bd None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:23:51.387002 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b792f1-cc79-415f-b5ae-b723824bc0bd None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:23:51.387002 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b792f1-cc79-415f-b5ae-b723824bc0bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:23:51.431273 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:23:51.454235 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-b8b792f1-cc79-415f-b5ae-b723824bc0bd None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:23:51.455513 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b8b792f1-cc79-415f-b5ae-b723824bc0bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:23:51.522930 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1222/2428] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:23:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:23:51.531442 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b3b3c8-9ffb-4c31-9798-8b5b03af3cc7 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:23:51.531749 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b3b3c8-9ffb-4c31-9798-8b5b03af3cc7 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:23:51.532017 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b3b3c8-9ffb-4c31-9798-8b5b03af3cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:23:51.553952 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-d6b3b3c8-9ffb-4c31-9798-8b5b03af3cc7 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:23:51.555561 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d6b3b3c8-9ffb-4c31-9798-8b5b03af3cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:23:51.564536 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1207/2429] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:23:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:23:51.569243 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b792f1-cc79-415f-b5ae-b723824bc0bd None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:23:51.580256 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b792f1-cc79-415f-b5ae-b723824bc0bd None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:23:51.585830 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30347a9-56f3-424d-8855-eead1462a254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:23:51.586449 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30347a9-56f3-424d-8855-eead1462a254 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:23:51.586724 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30347a9-56f3-424d-8855-eead1462a254 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:23:51.586992 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30347a9-56f3-424d-8855-eead1462a254 admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:23:51.587652 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30347a9-56f3-424d-8855-eead1462a254 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:23:51.587950 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30347a9-56f3-424d-8855-eead1462a254 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:23:51.589623 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30347a9-56f3-424d-8855-eead1462a254 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:23:51.602784 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1223/2430] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 6 16:23:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:23:51.607373 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b3b3c8-9ffb-4c31-9798-8b5b03af3cc7 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:23:51.617971 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b3b3c8-9ffb-4c31-9798-8b5b03af3cc7 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:23:51.623659 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47972bf-a52b-4da3-b765-21ba69639b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:23:51.624273 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47972bf-a52b-4da3-b765-21ba69639b6e admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:23:51.624543 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47972bf-a52b-4da3-b765-21ba69639b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:23:51.624845 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47972bf-a52b-4da3-b765-21ba69639b6e admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:23:51.625478 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47972bf-a52b-4da3-b765-21ba69639b6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:23:51.627385 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47972bf-a52b-4da3-b765-21ba69639b6e admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:23:51.659974 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1208/2431] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:23:51 2022] POST /identity/v3/projects => generated 371 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:23:51.664596 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30347a9-56f3-424d-8855-eead1462a254 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:23:51.675778 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30347a9-56f3-424d-8855-eead1462a254 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:23:51.739952 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332d24cb-155f-47de-8289-3b86f0a25b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:23:51.740572 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332d24cb-155f-47de-8289-3b86f0a25b3d admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:23:51.740876 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332d24cb-155f-47de-8289-3b86f0a25b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:23:51.741146 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332d24cb-155f-47de-8289-3b86f0a25b3d admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:23:51.741809 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332d24cb-155f-47de-8289-3b86f0a25b3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:23:51.743805 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332d24cb-155f-47de-8289-3b86f0a25b3d admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:23:51.765909 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1224/2432] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:23:51 2022] POST /identity/v3/users => generated 286 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:23:51.770164 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47972bf-a52b-4da3-b765-21ba69639b6e admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:23:51.781272 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47972bf-a52b-4da3-b765-21ba69639b6e admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:23:51.787047 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf54437e-8c78-49ed-ae7f-c9b1a1c47681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:23:51.787672 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf54437e-8c78-49ed-ae7f-c9b1a1c47681 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:23:51.787939 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf54437e-8c78-49ed-ae7f-c9b1a1c47681 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:23:51.788198 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf54437e-8c78-49ed-ae7f-c9b1a1c47681 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:23:51.789043 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf54437e-8c78-49ed-ae7f-c9b1a1c47681 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:23:51.790645 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf54437e-8c78-49ed-ae7f-c9b1a1c47681 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:23:51.801841 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1209/2433] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:23:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:23:51.806712 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332d24cb-155f-47de-8289-3b86f0a25b3d admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:23:51.817842 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332d24cb-155f-47de-8289-3b86f0a25b3d admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:23:51.822710 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72c95fd-44de-4bde-ae89-029467846285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:23:51.823311 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c95fd-44de-4bde-ae89-029467846285 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:23:51.823582 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c95fd-44de-4bde-ae89-029467846285 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:23:51.823846 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c95fd-44de-4bde-ae89-029467846285 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:23:51.824502 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72c95fd-44de-4bde-ae89-029467846285 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:23:51.826146 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72c95fd-44de-4bde-ae89-029467846285 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:23:51.838234 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1225/2434] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:23:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:23:51.842775 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf54437e-8c78-49ed-ae7f-c9b1a1c47681 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:23:51.853403 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf54437e-8c78-49ed-ae7f-c9b1a1c47681 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:23:51.858813 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7da0b27-6c39-43c9-809b-1001fca6500e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:23:51.859472 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7da0b27-6c39-43c9-809b-1001fca6500e admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:23:51.859776 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7da0b27-6c39-43c9-809b-1001fca6500e admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:23:51.860030 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7da0b27-6c39-43c9-809b-1001fca6500e admin admin] PATH_INFO: `/v3/projects/71b0e99f20e842c29ec46e5731ae9d9a/users/e8bab2b0d23a47e19c4a50e72fc75714/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:23:51.876950 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7da0b27-6c39-43c9-809b-1001fca6500e admin admin] RBAC: Authorizing `identity:create_grant(project_id=71b0e99f20e842c29ec46e5731ae9d9a, user_id=e8bab2b0d23a47e19c4a50e72fc75714, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:23:51.879600 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7da0b27-6c39-43c9-809b-1001fca6500e admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:23:51.897524 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1210/2435] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:23:51 2022] PUT /identity/v3/projects/71b0e99f20e842c29ec46e5731ae9d9a/users/e8bab2b0d23a47e19c4a50e72fc75714/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:23:51.903001 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8964878c-b0e7-499d-8d6e-188e1d4ba096 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:23:51.903459 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8964878c-b0e7-499d-8d6e-188e1d4ba096 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:23:51.903910 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8964878c-b0e7-499d-8d6e-188e1d4ba096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:23:51.924953 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-8964878c-b0e7-499d-8d6e-188e1d4ba096 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:23:51.926180 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-8964878c-b0e7-499d-8d6e-188e1d4ba096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:23:51.996027 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1226/2436] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:23:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:23:52.009487 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7da0b27-6c39-43c9-809b-1001fca6500e admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:23:52.011454 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d7da0b27-6c39-43c9-809b-1001fca6500e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:23:52.077173 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7da0b27-6c39-43c9-809b-1001fca6500e admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:23:52.081555 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7592b58b-7aee-402e-b7e7-89468ff3a8c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:23:52.082148 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7592b58b-7aee-402e-b7e7-89468ff3a8c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:23:52.082429 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7592b58b-7aee-402e-b7e7-89468ff3a8c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:23:52.082693 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7592b58b-7aee-402e-b7e7-89468ff3a8c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:23:52.093069 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7592b58b-7aee-402e-b7e7-89468ff3a8c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:23:52.094489 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7592b58b-7aee-402e-b7e7-89468ff3a8c8 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:23:52.104427 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1211/2437] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:23:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:23:56.456394 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a75315-66e6-4011-bb7e-bfb2f3241fb8 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:23:56.456924 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a75315-66e6-4011-bb7e-bfb2f3241fb8 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:23:56.457024 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a75315-66e6-4011-bb7e-bfb2f3241fb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:23:56.472403 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:23:56.501687 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-32a75315-66e6-4011-bb7e-bfb2f3241fb8 None None] MFA Rules not processed for user `e8bab2b0d23a47e19c4a50e72fc75714`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:23:56.504896 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-32a75315-66e6-4011-bb7e-bfb2f3241fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:23:56.592802 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1227/2438] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:23:56 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:23:56.595880 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7592b58b-7aee-402e-b7e7-89468ff3a8c8 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:23:56.607199 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7592b58b-7aee-402e-b7e7-89468ff3a8c8 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:23:56.612237 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03d41cf-bcd3-48d4-940a-523601611052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:23:56.613012 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03d41cf-bcd3-48d4-940a-523601611052 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:23:56.613356 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03d41cf-bcd3-48d4-940a-523601611052 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:23:56.613638 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03d41cf-bcd3-48d4-940a-523601611052 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:23:56.614368 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03d41cf-bcd3-48d4-940a-523601611052 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:23:56.616402 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03d41cf-bcd3-48d4-940a-523601611052 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:23:56.649340 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1212/2439] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:23:56 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:23:56.653220 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a75315-66e6-4011-bb7e-bfb2f3241fb8 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:23:56.665143 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a75315-66e6-4011-bb7e-bfb2f3241fb8 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:23:56.741664 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8275b0e0-e79f-4a2c-9eee-765ebe12f777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:23:56.742307 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8275b0e0-e79f-4a2c-9eee-765ebe12f777 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:23:56.742587 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8275b0e0-e79f-4a2c-9eee-765ebe12f777 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:23:56.742857 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8275b0e0-e79f-4a2c-9eee-765ebe12f777 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:23:56.743910 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8275b0e0-e79f-4a2c-9eee-765ebe12f777 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:23:56.746125 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8275b0e0-e79f-4a2c-9eee-765ebe12f777 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:23:56.772928 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1228/2440] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:23:56 2022] POST /identity/v3/users => generated 293 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:23:56.776311 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03d41cf-bcd3-48d4-940a-523601611052 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:23:56.789583 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03d41cf-bcd3-48d4-940a-523601611052 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:23:56.794737 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7053f2-fbea-44cb-bf9c-0cdd353763db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:23:56.795519 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7053f2-fbea-44cb-bf9c-0cdd353763db admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:23:56.795806 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7053f2-fbea-44cb-bf9c-0cdd353763db admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:23:56.796075 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7053f2-fbea-44cb-bf9c-0cdd353763db admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:23:56.796741 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7053f2-fbea-44cb-bf9c-0cdd353763db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:23:56.798444 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7053f2-fbea-44cb-bf9c-0cdd353763db admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:23:56.819503 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1213/2441] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:23:56 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 16:23:56.821503 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8275b0e0-e79f-4a2c-9eee-765ebe12f777 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:23:56.835950 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8275b0e0-e79f-4a2c-9eee-765ebe12f777 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:23:56.841732 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79367fd5-94ab-4cd7-a8ca-6f854fb48a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:23:56.842433 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79367fd5-94ab-4cd7-a8ca-6f854fb48a55 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:23:56.842708 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79367fd5-94ab-4cd7-a8ca-6f854fb48a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:23:56.842977 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79367fd5-94ab-4cd7-a8ca-6f854fb48a55 admin admin] PATH_INFO: `/v3/projects/b8ed1d1252d1476987a7a34dfaceb395/users/0289d3009cd549829062fdeb06ff39df/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:23:56.859859 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79367fd5-94ab-4cd7-a8ca-6f854fb48a55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8ed1d1252d1476987a7a34dfaceb395, user_id=0289d3009cd549829062fdeb06ff39df, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:23:56.862665 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79367fd5-94ab-4cd7-a8ca-6f854fb48a55 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:23:56.888165 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1229/2442] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:23:56 2022] PUT /identity/v3/projects/b8ed1d1252d1476987a7a34dfaceb395/users/0289d3009cd549829062fdeb06ff39df/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:23:56.889490 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7053f2-fbea-44cb-bf9c-0cdd353763db admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:23:56.899794 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7053f2-fbea-44cb-bf9c-0cdd353763db admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:23:56.954767 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:23:56.966208 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382c1e2e-2214-40dd-96cc-be4494236adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:23:56.966849 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382c1e2e-2214-40dd-96cc-be4494236adf admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:23:56.967831 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382c1e2e-2214-40dd-96cc-be4494236adf admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:23:56.968242 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382c1e2e-2214-40dd-96cc-be4494236adf admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:23:56.969378 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382c1e2e-2214-40dd-96cc-be4494236adf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:23:56.971188 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382c1e2e-2214-40dd-96cc-be4494236adf admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:23:56.982694 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1214/2443] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:23:56 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:23:56.987260 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79367fd5-94ab-4cd7-a8ca-6f854fb48a55 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:23:57.000483 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79367fd5-94ab-4cd7-a8ca-6f854fb48a55 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:23:57.006490 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d402365-b1af-4ceb-8576-dac16c33992d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:23:57.007690 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d402365-b1af-4ceb-8576-dac16c33992d admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:23:57.008175 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d402365-b1af-4ceb-8576-dac16c33992d admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:23:57.008612 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d402365-b1af-4ceb-8576-dac16c33992d admin admin] PATH_INFO: `/v3/projects/b8ed1d1252d1476987a7a34dfaceb395/users/0289d3009cd549829062fdeb06ff39df/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:23:57.011402 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d402365-b1af-4ceb-8576-dac16c33992d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8ed1d1252d1476987a7a34dfaceb395, user_id=0289d3009cd549829062fdeb06ff39df, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:23:57.014586 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d402365-b1af-4ceb-8576-dac16c33992d admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:23:57.026837 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1230/2444] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:23:56 2022] PUT /identity/v3/projects/b8ed1d1252d1476987a7a34dfaceb395/users/0289d3009cd549829062fdeb06ff39df/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:24:01.614484 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d651ce-9961-42bf-be9d-470452812cb7 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:24:01.614989 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d651ce-9961-42bf-be9d-470452812cb7 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:24:01.615159 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d651ce-9961-42bf-be9d-470452812cb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:24:01.650416 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-d0d651ce-9961-42bf-be9d-470452812cb7 None None] MFA Rules not processed for user `0289d3009cd549829062fdeb06ff39df`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:24:01.651633 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d0d651ce-9961-42bf-be9d-470452812cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:24:01.740817 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1215/2445] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:24:01 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:24:01.754578 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d402365-b1af-4ceb-8576-dac16c33992d admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:24:01.756043 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-2d402365-b1af-4ceb-8576-dac16c33992d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:24:01.820642 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:24:01.877575 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d402365-b1af-4ceb-8576-dac16c33992d admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:24:01.883360 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5abb8c1-8f28-4f09-a5df-5e9315393a3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63de41f4348243998029f7d07aa5dc40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:24:01.884038 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5abb8c1-8f28-4f09-a5df-5e9315393a3a service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:24:01.884313 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5abb8c1-8f28-4f09-a5df-5e9315393a3a service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:24:01.884581 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5abb8c1-8f28-4f09-a5df-5e9315393a3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:24:01.945920 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5abb8c1-8f28-4f09-a5df-5e9315393a3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:24:01.947628 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5abb8c1-8f28-4f09-a5df-5e9315393a3a service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:24:01.957371 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1231/2446] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:24:01 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:24:03.014385 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e5597d-54d0-45ca-a363-4ec5783cad89 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:24:03.014913 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e5597d-54d0-45ca-a363-4ec5783cad89 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:24:03.014913 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e5597d-54d0-45ca-a363-4ec5783cad89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:24:03.034946 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-89e5597d-54d0-45ca-a363-4ec5783cad89 None None] MFA Rules not processed for user `e8bab2b0d23a47e19c4a50e72fc75714`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:24:03.036217 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-89e5597d-54d0-45ca-a363-4ec5783cad89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:24:03.043900 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1216/2447] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:24:03 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:24:11.952817 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=47 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:24:13.029635 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=40 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:24:28.204445 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5abb8c1-8f28-4f09-a5df-5e9315393a3a service nova] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:24:28.206039 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e5abb8c1-8f28-4f09-a5df-5e9315393a3a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:24:28.301084 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5abb8c1-8f28-4f09-a5df-5e9315393a3a service nova] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:24:28.306895 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c1938f-b1a8-4ac0-83f9-53264942897f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71c2912a44f04bd983b4d4112dbee734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:24:28.307500 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c1938f-b1a8-4ac0-83f9-53264942897f service placement] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:24:28.307774 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c1938f-b1a8-4ac0-83f9-53264942897f service placement] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:24:28.308044 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c1938f-b1a8-4ac0-83f9-53264942897f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:24:28.310015 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-87c1938f-b1a8-4ac0-83f9-53264942897f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:24:28.321544 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c1938f-b1a8-4ac0-83f9-53264942897f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:24:28.323121 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c1938f-b1a8-4ac0-83f9-53264942897f service placement] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:24:28.332961 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1232/2448] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:24:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:24:38.328703 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=41 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:24:40.823666 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051c9a54-46e6-4413-8b73-a533ccfe9d18 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:24:40.824195 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051c9a54-46e6-4413-8b73-a533ccfe9d18 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:24:40.824195 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051c9a54-46e6-4413-8b73-a533ccfe9d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:24:40.846870 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-051c9a54-46e6-4413-8b73-a533ccfe9d18 None None] MFA Rules not processed for user `e225aec64aee47449542aecbec0ae561`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:24:40.848454 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-051c9a54-46e6-4413-8b73-a533ccfe9d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:24:40.906610 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1217/2449] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:24:40 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:24:43.458219 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c1938f-b1a8-4ac0-83f9-53264942897f service placement] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:24:43.459766 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-87c1938f-b1a8-4ac0-83f9-53264942897f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:24:43.539625 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c1938f-b1a8-4ac0-83f9-53264942897f service placement] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:24:43.543804 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4ade5b-7cf4-4207-aae4-7d7cef039468 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:24:43.544403 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ade5b-7cf4-4207-aae4-7d7cef039468 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:24:43.544682 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ade5b-7cf4-4207-aae4-7d7cef039468 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:24:43.544975 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ade5b-7cf4-4207-aae4-7d7cef039468 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:24:43.554478 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4ade5b-7cf4-4207-aae4-7d7cef039468 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:24:43.555920 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4ade5b-7cf4-4207-aae4-7d7cef039468 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:24:43.565443 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1233/2450] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:24:43 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:24:50.899972 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=25 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:24:53.561425 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=35 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:24:56.451140 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051c9a54-46e6-4413-8b73-a533ccfe9d18 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:24:56.461081 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051c9a54-46e6-4413-8b73-a533ccfe9d18 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:24:56.464869 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f3d511-0e40-42d5-8ef7-b6a490b144b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:24:56.465480 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f3d511-0e40-42d5-8ef7-b6a490b144b2 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:24:56.465757 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f3d511-0e40-42d5-8ef7-b6a490b144b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:24:56.466036 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f3d511-0e40-42d5-8ef7-b6a490b144b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:24:56.467910 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-36f3d511-0e40-42d5-8ef7-b6a490b144b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:24:56.533256 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f3d511-0e40-42d5-8ef7-b6a490b144b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:24:56.534699 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f3d511-0e40-42d5-8ef7-b6a490b144b2 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:24:56.544797 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1218/2451] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:24:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:24:59.963326 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4ade5b-7cf4-4207-aae4-7d7cef039468 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:24:59.964884 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ee4ade5b-7cf4-4207-aae4-7d7cef039468 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:24:59.980144 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4ade5b-7cf4-4207-aae4-7d7cef039468 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:24:59.984892 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5506b108-804a-416c-bfcc-11f4a1c6a06c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:24:59.985812 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5506b108-804a-416c-bfcc-11f4a1c6a06c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:24:59.986094 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5506b108-804a-416c-bfcc-11f4a1c6a06c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:24:59.986412 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5506b108-804a-416c-bfcc-11f4a1c6a06c admin admin] PATH_INFO: `/v3/users/93096a58fcb34f2085a05ffa40df5937` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:24:59.987562 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5506b108-804a-416c-bfcc-11f4a1c6a06c admin admin] RBAC: Authorizing `identity:delete_user(user_id=93096a58fcb34f2085a05ffa40df5937)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:24:59.989546 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5506b108-804a-416c-bfcc-11f4a1c6a06c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:00.047806 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5506b108-804a-416c-bfcc-11f4a1c6a06c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93096a58fcb34f2085a05ffa40df5937'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:00.047806 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5506b108-804a-416c-bfcc-11f4a1c6a06c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93096a58fcb34f2085a05ffa40df5937'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:00.060444 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5506b108-804a-416c-bfcc-11f4a1c6a06c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93096a58fcb34f2085a05ffa40df5937'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:00.075230 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5506b108-804a-416c-bfcc-11f4a1c6a06c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93096a58fcb34f2085a05ffa40df5937'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:00.076052 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5506b108-804a-416c-bfcc-11f4a1c6a06c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93096a58fcb34f2085a05ffa40df5937'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:00.090793 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1234/2452] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:24:59 2022] DELETE /identity/v3/users/93096a58fcb34f2085a05ffa40df5937 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:25:00.289725 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f3d511-0e40-42d5-8ef7-b6a490b144b2 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:00.305505 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f3d511-0e40-42d5-8ef7-b6a490b144b2 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:00.391642 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abafb81b-6c92-4150-b625-a30591c48687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:00.392303 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abafb81b-6c92-4150-b625-a30591c48687 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:00.392605 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abafb81b-6c92-4150-b625-a30591c48687 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:00.392899 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abafb81b-6c92-4150-b625-a30591c48687 admin admin] PATH_INFO: `/v3/projects/49a3e0cfba224a119bf8aaf26ae583f4` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:00.394169 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abafb81b-6c92-4150-b625-a30591c48687 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49a3e0cfba224a119bf8aaf26ae583f4)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:00.397056 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abafb81b-6c92-4150-b625-a30591c48687 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:00.444799 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-abafb81b-6c92-4150-b625-a30591c48687 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49a3e0cfba224a119bf8aaf26ae583f4', 'request_id': 'req-abafb81b-6c92-4150-b625-a30591c48687'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:00.455741 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-abafb81b-6c92-4150-b625-a30591c48687 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49a3e0cfba224a119bf8aaf26ae583f4', 'request_id': 'req-abafb81b-6c92-4150-b625-a30591c48687'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:00.463070 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-abafb81b-6c92-4150-b625-a30591c48687 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49a3e0cfba224a119bf8aaf26ae583f4', 'request_id': 'req-abafb81b-6c92-4150-b625-a30591c48687'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:00.476129 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-abafb81b-6c92-4150-b625-a30591c48687 admin admin] The token cache is being invalidate because project 49a3e0cfba224a119bf8aaf26ae583f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:25:00.476339 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-abafb81b-6c92-4150-b625-a30591c48687 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:00.477118 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-abafb81b-6c92-4150-b625-a30591c48687 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:00.478901 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1219/2453] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:25:00 2022] DELETE /identity/v3/projects/49a3e0cfba224a119bf8aaf26ae583f4 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:25:00.484154 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65802e16-3269-4d85-82e0-20a36923ac33 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:00.484533 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65802e16-3269-4d85-82e0-20a36923ac33 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:00.484891 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65802e16-3269-4d85-82e0-20a36923ac33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:00.505598 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-65802e16-3269-4d85-82e0-20a36923ac33 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:25:00.506856 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-65802e16-3269-4d85-82e0-20a36923ac33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:00.567664 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1235/2454] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:25:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:25:00.576367 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35c084c-631d-4016-8ca6-54fd52c93188 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:00.576643 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35c084c-631d-4016-8ca6-54fd52c93188 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:00.576942 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35c084c-631d-4016-8ca6-54fd52c93188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:00.597234 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-f35c084c-631d-4016-8ca6-54fd52c93188 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:25:00.598462 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f35c084c-631d-4016-8ca6-54fd52c93188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:00.606512 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1220/2455] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:25:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:25:00.610603 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65802e16-3269-4d85-82e0-20a36923ac33 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:00.621543 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65802e16-3269-4d85-82e0-20a36923ac33 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:00.626873 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7f04b2-cd55-4a3d-850d-9d90a611825f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:00.627529 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7f04b2-cd55-4a3d-850d-9d90a611825f admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:00.627809 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7f04b2-cd55-4a3d-850d-9d90a611825f admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:00.628083 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7f04b2-cd55-4a3d-850d-9d90a611825f admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:00.628796 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7f04b2-cd55-4a3d-850d-9d90a611825f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:25:00.629065 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7f04b2-cd55-4a3d-850d-9d90a611825f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:00.630780 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7f04b2-cd55-4a3d-850d-9d90a611825f admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:00.647821 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1236/2456] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 6 16:25:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:25:00.649159 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35c084c-631d-4016-8ca6-54fd52c93188 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:00.662611 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35c084c-631d-4016-8ca6-54fd52c93188 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:00.668118 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d21643-c32f-4a7f-aaec-04aefba42f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:00.668735 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d21643-c32f-4a7f-aaec-04aefba42f75 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:00.669063 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d21643-c32f-4a7f-aaec-04aefba42f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:00.669467 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d21643-c32f-4a7f-aaec-04aefba42f75 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:00.670399 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d21643-c32f-4a7f-aaec-04aefba42f75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:00.672036 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d21643-c32f-4a7f-aaec-04aefba42f75 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:00.690246 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:25:00.705983 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1221/2457] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:25:00 2022] POST /identity/v3/projects => generated 385 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:25:00.710269 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7f04b2-cd55-4a3d-850d-9d90a611825f admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:00.720993 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7f04b2-cd55-4a3d-850d-9d90a611825f admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:00.780810 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bf384a-6252-45a0-a7fc-d2bc25cea5b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:00.781406 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf384a-6252-45a0-a7fc-d2bc25cea5b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:00.781686 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf384a-6252-45a0-a7fc-d2bc25cea5b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:00.781951 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf384a-6252-45a0-a7fc-d2bc25cea5b9 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:00.782584 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bf384a-6252-45a0-a7fc-d2bc25cea5b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:00.784541 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bf384a-6252-45a0-a7fc-d2bc25cea5b9 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:00.792879 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:25:00.813476 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1237/2458] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:25:00 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:25:00.814848 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d21643-c32f-4a7f-aaec-04aefba42f75 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:00.825964 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d21643-c32f-4a7f-aaec-04aefba42f75 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:00.830805 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0a1e3a-701c-41b1-804a-05292a08c34b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:00.831405 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0a1e3a-701c-41b1-804a-05292a08c34b admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:00.831679 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0a1e3a-701c-41b1-804a-05292a08c34b admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:00.831948 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0a1e3a-701c-41b1-804a-05292a08c34b admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:00.832626 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0a1e3a-701c-41b1-804a-05292a08c34b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:00.834263 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0a1e3a-701c-41b1-804a-05292a08c34b admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:00.845580 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1222/2459] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:25:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:25:00.849778 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bf384a-6252-45a0-a7fc-d2bc25cea5b9 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:00.860876 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bf384a-6252-45a0-a7fc-d2bc25cea5b9 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:00.865807 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7fd5f1-f722-457f-8252-ff4e6f5a4af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:00.866424 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7fd5f1-f722-457f-8252-ff4e6f5a4af0 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:00.866695 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7fd5f1-f722-457f-8252-ff4e6f5a4af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:00.866963 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7fd5f1-f722-457f-8252-ff4e6f5a4af0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:00.867614 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7fd5f1-f722-457f-8252-ff4e6f5a4af0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:00.869331 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7fd5f1-f722-457f-8252-ff4e6f5a4af0 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:00.880895 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1238/2460] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:25:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:25:00.885166 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0a1e3a-701c-41b1-804a-05292a08c34b admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:00.895272 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0a1e3a-701c-41b1-804a-05292a08c34b admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:00.900145 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57149fc-777c-444f-8590-f05d86ebd918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:00.900810 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57149fc-777c-444f-8590-f05d86ebd918 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:00.901087 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57149fc-777c-444f-8590-f05d86ebd918 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:00.901355 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57149fc-777c-444f-8590-f05d86ebd918 admin admin] PATH_INFO: `/v3/projects/4b9d52517f2145b6ac1cf64831658783/users/a1d69be709e647fbb3a85c447daa8f2a/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:00.916510 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57149fc-777c-444f-8590-f05d86ebd918 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b9d52517f2145b6ac1cf64831658783, user_id=a1d69be709e647fbb3a85c447daa8f2a, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:00.919181 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57149fc-777c-444f-8590-f05d86ebd918 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:00.934347 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1223/2461] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:25:00 2022] PUT /identity/v3/projects/4b9d52517f2145b6ac1cf64831658783/users/a1d69be709e647fbb3a85c447daa8f2a/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:25:00.944663 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adfa83d-29e4-4561-83cd-e846a6d2a437 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:00.945096 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adfa83d-29e4-4561-83cd-e846a6d2a437 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:00.945413 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adfa83d-29e4-4561-83cd-e846a6d2a437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:00.974400 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-1adfa83d-29e4-4561-83cd-e846a6d2a437 None None] MFA Rules not processed for user `a1d69be709e647fbb3a85c447daa8f2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:25:00.975626 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1adfa83d-29e4-4561-83cd-e846a6d2a437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:01.057317 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1239/2462] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:25:00 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:25:01.064720 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57149fc-777c-444f-8590-f05d86ebd918 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:01.066662 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e57149fc-777c-444f-8590-f05d86ebd918 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:01.148035 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57149fc-777c-444f-8590-f05d86ebd918 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:01.154086 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd24f0e-3bf5-48d2-8f18-cb4ababb23bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63de41f4348243998029f7d07aa5dc40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:01.154707 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd24f0e-3bf5-48d2-8f18-cb4ababb23bb service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:01.154995 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd24f0e-3bf5-48d2-8f18-cb4ababb23bb service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:01.155257 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd24f0e-3bf5-48d2-8f18-cb4ababb23bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:01.164085 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd24f0e-3bf5-48d2-8f18-cb4ababb23bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:01.165731 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd24f0e-3bf5-48d2-8f18-cb4ababb23bb service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:01.175956 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1224/2463] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:25:01 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:25:10.747134 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adfa83d-29e4-4561-83cd-e846a6d2a437 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:10.812987 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adfa83d-29e4-4561-83cd-e846a6d2a437 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:10.818840 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df1fc7f-1ff0-4a8f-a295-5315da080faf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:10.819482 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df1fc7f-1ff0-4a8f-a295-5315da080faf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:10.819764 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df1fc7f-1ff0-4a8f-a295-5315da080faf admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:10.820035 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df1fc7f-1ff0-4a8f-a295-5315da080faf admin admin] PATH_INFO: `/v3/users/a1d69be709e647fbb3a85c447daa8f2a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:10.821257 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df1fc7f-1ff0-4a8f-a295-5315da080faf admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1d69be709e647fbb3a85c447daa8f2a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:10.823209 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df1fc7f-1ff0-4a8f-a295-5315da080faf admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:10.871051 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2df1fc7f-1ff0-4a8f-a295-5315da080faf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1d69be709e647fbb3a85c447daa8f2a'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:10.871641 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2df1fc7f-1ff0-4a8f-a295-5315da080faf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1d69be709e647fbb3a85c447daa8f2a'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:10.878673 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2df1fc7f-1ff0-4a8f-a295-5315da080faf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1d69be709e647fbb3a85c447daa8f2a'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:10.888626 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:INSERT=3,keystone:SELECT=89,keystone:DELETE=8 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:25:10.891449 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2df1fc7f-1ff0-4a8f-a295-5315da080faf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1d69be709e647fbb3a85c447daa8f2a'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:10.892528 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2df1fc7f-1ff0-4a8f-a295-5315da080faf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1d69be709e647fbb3a85c447daa8f2a'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:10.906608 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1240/2464] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:25:10 2022] DELETE /identity/v3/users/a1d69be709e647fbb3a85c447daa8f2a => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:25:10.911252 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4532fc8c-10b3-4bca-9d6f-926d35c3f0e8 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:10.911742 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4532fc8c-10b3-4bca-9d6f-926d35c3f0e8 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:10.911939 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4532fc8c-10b3-4bca-9d6f-926d35c3f0e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:10.933040 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-4532fc8c-10b3-4bca-9d6f-926d35c3f0e8 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:25:10.934255 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-4532fc8c-10b3-4bca-9d6f-926d35c3f0e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:11.001210 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1225/2465] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:25:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:25:11.011716 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df1fc7f-1ff0-4a8f-a295-5315da080faf admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:11.013346 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-2df1fc7f-1ff0-4a8f-a295-5315da080faf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:11.082477 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df1fc7f-1ff0-4a8f-a295-5315da080faf admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:11.086423 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12862238-86cd-4ac6-9467-eed9f5968961 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:11.087040 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12862238-86cd-4ac6-9467-eed9f5968961 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:11.087339 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12862238-86cd-4ac6-9467-eed9f5968961 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:11.087605 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12862238-86cd-4ac6-9467-eed9f5968961 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:11.097668 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12862238-86cd-4ac6-9467-eed9f5968961 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:11.099118 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12862238-86cd-4ac6-9467-eed9f5968961 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:11.109263 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1241/2466] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:25:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:25:11.395240 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4532fc8c-10b3-4bca-9d6f-926d35c3f0e8 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:11.396688 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-4532fc8c-10b3-4bca-9d6f-926d35c3f0e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:11.410450 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4532fc8c-10b3-4bca-9d6f-926d35c3f0e8 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:11.415701 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43d4bd2-1dcc-4bb4-8df9-a6390bd7fcbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:11.417124 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d4bd2-1dcc-4bb4-8df9-a6390bd7fcbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:11.417439 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d4bd2-1dcc-4bb4-8df9-a6390bd7fcbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:11.417885 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d4bd2-1dcc-4bb4-8df9-a6390bd7fcbf admin admin] PATH_INFO: `/v3/projects/4b9d52517f2145b6ac1cf64831658783` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:11.419049 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43d4bd2-1dcc-4bb4-8df9-a6390bd7fcbf admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b9d52517f2145b6ac1cf64831658783)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:11.420980 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43d4bd2-1dcc-4bb4-8df9-a6390bd7fcbf admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:11.465855 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:25:11.474339 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e43d4bd2-1dcc-4bb4-8df9-a6390bd7fcbf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b9d52517f2145b6ac1cf64831658783', 'request_id': 'req-e43d4bd2-1dcc-4bb4-8df9-a6390bd7fcbf'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:11.484854 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e43d4bd2-1dcc-4bb4-8df9-a6390bd7fcbf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b9d52517f2145b6ac1cf64831658783', 'request_id': 'req-e43d4bd2-1dcc-4bb4-8df9-a6390bd7fcbf'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:11.491644 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e43d4bd2-1dcc-4bb4-8df9-a6390bd7fcbf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b9d52517f2145b6ac1cf64831658783', 'request_id': 'req-e43d4bd2-1dcc-4bb4-8df9-a6390bd7fcbf'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:11.503799 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e43d4bd2-1dcc-4bb4-8df9-a6390bd7fcbf admin admin] The token cache is being invalidate because project 4b9d52517f2145b6ac1cf64831658783 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:25:11.504129 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e43d4bd2-1dcc-4bb4-8df9-a6390bd7fcbf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:11.504946 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e43d4bd2-1dcc-4bb4-8df9-a6390bd7fcbf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:11.506641 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1226/2467] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:25:11 2022] DELETE /identity/v3/projects/4b9d52517f2145b6ac1cf64831658783 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:25:11.511999 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d8550c-d8cb-42d3-9e85-ef6b1e89a114 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:11.512271 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d8550c-d8cb-42d3-9e85-ef6b1e89a114 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:11.512567 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d8550c-d8cb-42d3-9e85-ef6b1e89a114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:11.536170 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-37d8550c-d8cb-42d3-9e85-ef6b1e89a114 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:25:11.537501 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-37d8550c-d8cb-42d3-9e85-ef6b1e89a114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:11.608532 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1242/2468] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:25:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:25:11.617328 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dae0cc-1eba-4b76-b324-953e71f35519 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:11.617605 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dae0cc-1eba-4b76-b324-953e71f35519 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:11.617867 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dae0cc-1eba-4b76-b324-953e71f35519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:11.639676 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-f1dae0cc-1eba-4b76-b324-953e71f35519 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:25:11.640923 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f1dae0cc-1eba-4b76-b324-953e71f35519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:11.650402 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1227/2469] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:25:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:25:11.655489 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d8550c-d8cb-42d3-9e85-ef6b1e89a114 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:11.666854 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d8550c-d8cb-42d3-9e85-ef6b1e89a114 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:11.673159 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4899879-c4b0-4d27-af07-2354ccdbbf94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:11.673816 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4899879-c4b0-4d27-af07-2354ccdbbf94 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:11.674093 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4899879-c4b0-4d27-af07-2354ccdbbf94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:11.674356 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4899879-c4b0-4d27-af07-2354ccdbbf94 admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:11.675016 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4899879-c4b0-4d27-af07-2354ccdbbf94 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:25:11.675317 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4899879-c4b0-4d27-af07-2354ccdbbf94 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:11.677014 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4899879-c4b0-4d27-af07-2354ccdbbf94 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:11.691756 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1243/2470] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 6 16:25:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:25:11.697569 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dae0cc-1eba-4b76-b324-953e71f35519 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:11.708232 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dae0cc-1eba-4b76-b324-953e71f35519 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:11.714389 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d4928e-9f9a-4e90-b9f2-0947d2cb9563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:11.715978 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d4928e-9f9a-4e90-b9f2-0947d2cb9563 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:11.715978 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d4928e-9f9a-4e90-b9f2-0947d2cb9563 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:11.715978 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d4928e-9f9a-4e90-b9f2-0947d2cb9563 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:11.717430 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d4928e-9f9a-4e90-b9f2-0947d2cb9563 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:11.719268 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d4928e-9f9a-4e90-b9f2-0947d2cb9563 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:11.752236 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1228/2471] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:25:11 2022] POST /identity/v3/projects => generated 389 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:25:11.756865 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4899879-c4b0-4d27-af07-2354ccdbbf94 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:11.768360 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4899879-c4b0-4d27-af07-2354ccdbbf94 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:11.835472 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bb14c5-f7ae-495c-aeb3-d33f7bad95b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:11.836092 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb14c5-f7ae-495c-aeb3-d33f7bad95b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:11.836370 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb14c5-f7ae-495c-aeb3-d33f7bad95b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:11.836636 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb14c5-f7ae-495c-aeb3-d33f7bad95b4 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:11.837300 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bb14c5-f7ae-495c-aeb3-d33f7bad95b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:11.839248 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bb14c5-f7ae-495c-aeb3-d33f7bad95b4 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:11.864542 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1244/2472] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:25:11 2022] POST /identity/v3/users => generated 295 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:25:11.866486 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d4928e-9f9a-4e90-b9f2-0947d2cb9563 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:11.879621 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d4928e-9f9a-4e90-b9f2-0947d2cb9563 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:11.886070 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec037736-e3e2-4f1c-a147-89817dd22706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:11.886789 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec037736-e3e2-4f1c-a147-89817dd22706 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:11.886939 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec037736-e3e2-4f1c-a147-89817dd22706 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:11.887214 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec037736-e3e2-4f1c-a147-89817dd22706 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:11.887883 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec037736-e3e2-4f1c-a147-89817dd22706 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:11.889544 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec037736-e3e2-4f1c-a147-89817dd22706 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:11.900063 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1229/2473] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:25:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:25:11.904373 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bb14c5-f7ae-495c-aeb3-d33f7bad95b4 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:11.914622 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bb14c5-f7ae-495c-aeb3-d33f7bad95b4 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:11.919361 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dc7572-1240-4e34-8062-4d0a458a7b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:11.920081 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc7572-1240-4e34-8062-4d0a458a7b72 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:11.920359 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc7572-1240-4e34-8062-4d0a458a7b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:11.920631 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc7572-1240-4e34-8062-4d0a458a7b72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:11.921323 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dc7572-1240-4e34-8062-4d0a458a7b72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:11.923000 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dc7572-1240-4e34-8062-4d0a458a7b72 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:11.924613 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:25:11.937676 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1245/2474] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:25:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:25:11.942083 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec037736-e3e2-4f1c-a147-89817dd22706 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:11.952534 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec037736-e3e2-4f1c-a147-89817dd22706 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:11.957237 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d435fef-dc25-4a86-8431-6958ddb53325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:11.957955 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d435fef-dc25-4a86-8431-6958ddb53325 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:11.958233 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d435fef-dc25-4a86-8431-6958ddb53325 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:11.958502 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d435fef-dc25-4a86-8431-6958ddb53325 admin admin] PATH_INFO: `/v3/projects/66d36e32b67442e8b754c92e12b693f0/users/3493bbdeab23450c80a22389e5ac78e7/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:11.974366 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d435fef-dc25-4a86-8431-6958ddb53325 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66d36e32b67442e8b754c92e12b693f0, user_id=3493bbdeab23450c80a22389e5ac78e7, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:11.977155 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d435fef-dc25-4a86-8431-6958ddb53325 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:11.992226 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1230/2475] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:25:11 2022] PUT /identity/v3/projects/66d36e32b67442e8b754c92e12b693f0/users/3493bbdeab23450c80a22389e5ac78e7/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:25:12.004182 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008f5f39-51bd-40b9-a732-b921ff1af162 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:12.004490 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008f5f39-51bd-40b9-a732-b921ff1af162 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:12.004838 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008f5f39-51bd-40b9-a732-b921ff1af162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:12.033506 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-008f5f39-51bd-40b9-a732-b921ff1af162 None None] MFA Rules not processed for user `3493bbdeab23450c80a22389e5ac78e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:25:12.034790 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-008f5f39-51bd-40b9-a732-b921ff1af162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:12.118313 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1246/2476] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:25:12 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:25:12.122794 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d435fef-dc25-4a86-8431-6958ddb53325 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:12.189331 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d435fef-dc25-4a86-8431-6958ddb53325 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:12.196892 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e748b9-ee9f-459e-bb55-57b28a26c4b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:12.197608 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e748b9-ee9f-459e-bb55-57b28a26c4b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:12.197886 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e748b9-ee9f-459e-bb55-57b28a26c4b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:12.198152 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e748b9-ee9f-459e-bb55-57b28a26c4b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:12.198801 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e748b9-ee9f-459e-bb55-57b28a26c4b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:12.200773 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e748b9-ee9f-459e-bb55-57b28a26c4b9 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:12.228570 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1231/2477] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:25:12 2022] POST /identity/v3/projects => generated 387 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:25:12.232894 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008f5f39-51bd-40b9-a732-b921ff1af162 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:12.243663 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008f5f39-51bd-40b9-a732-b921ff1af162 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:12.302054 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fa4cba-b6f4-4182-84c8-695140252c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:12.302666 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fa4cba-b6f4-4182-84c8-695140252c0f admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:12.303397 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fa4cba-b6f4-4182-84c8-695140252c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:12.303675 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fa4cba-b6f4-4182-84c8-695140252c0f admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:12.304297 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fa4cba-b6f4-4182-84c8-695140252c0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:12.306330 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fa4cba-b6f4-4182-84c8-695140252c0f admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:12.328358 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1247/2478] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:25:12 2022] POST /identity/v3/users => generated 300 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:25:12.332616 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e748b9-ee9f-459e-bb55-57b28a26c4b9 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:12.343403 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e748b9-ee9f-459e-bb55-57b28a26c4b9 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:12.348566 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fdd976-911c-4db3-b07d-98079e67dd28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:12.349198 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fdd976-911c-4db3-b07d-98079e67dd28 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:12.349473 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fdd976-911c-4db3-b07d-98079e67dd28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:12.349744 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fdd976-911c-4db3-b07d-98079e67dd28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:12.350406 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fdd976-911c-4db3-b07d-98079e67dd28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:12.352004 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fdd976-911c-4db3-b07d-98079e67dd28 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:12.362764 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1232/2479] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:25:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:25:12.367271 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fa4cba-b6f4-4182-84c8-695140252c0f admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:12.377854 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fa4cba-b6f4-4182-84c8-695140252c0f admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:12.383305 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbcc0ad-d1d9-4956-a827-460130cabc8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:12.383931 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbcc0ad-d1d9-4956-a827-460130cabc8a admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:12.384224 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbcc0ad-d1d9-4956-a827-460130cabc8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:12.384497 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbcc0ad-d1d9-4956-a827-460130cabc8a admin admin] PATH_INFO: `/v3/projects/3a84d026987343f4a029132a2abc61cd/users/7881d5d880764c43b5dbe6343de5be8a/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:12.399762 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbcc0ad-d1d9-4956-a827-460130cabc8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a84d026987343f4a029132a2abc61cd, user_id=7881d5d880764c43b5dbe6343de5be8a, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:12.402488 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbcc0ad-d1d9-4956-a827-460130cabc8a admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:12.416922 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1248/2480] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:25:12 2022] PUT /identity/v3/projects/3a84d026987343f4a029132a2abc61cd/users/7881d5d880764c43b5dbe6343de5be8a/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:25:12.421102 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fdd976-911c-4db3-b07d-98079e67dd28 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:12.431796 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fdd976-911c-4db3-b07d-98079e67dd28 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:12.470305 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:25:12.490008 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9488fc13-e81d-4533-84fd-d85d415bda53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:12.490940 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9488fc13-e81d-4533-84fd-d85d415bda53 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:12.490940 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9488fc13-e81d-4533-84fd-d85d415bda53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:12.491183 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9488fc13-e81d-4533-84fd-d85d415bda53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:12.491784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9488fc13-e81d-4533-84fd-d85d415bda53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:12.493427 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9488fc13-e81d-4533-84fd-d85d415bda53 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:12.503725 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1233/2481] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:25:12 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:25:12.508371 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbcc0ad-d1d9-4956-a827-460130cabc8a admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:12.519335 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbcc0ad-d1d9-4956-a827-460130cabc8a admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:12.525211 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5267d84c-cb41-4dce-9b7c-ce85fd8f15a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:12.525844 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5267d84c-cb41-4dce-9b7c-ce85fd8f15a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:12.526132 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5267d84c-cb41-4dce-9b7c-ce85fd8f15a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:12.526399 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5267d84c-cb41-4dce-9b7c-ce85fd8f15a5 admin admin] PATH_INFO: `/v3/projects/3a84d026987343f4a029132a2abc61cd/users/7881d5d880764c43b5dbe6343de5be8a/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:12.528544 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5267d84c-cb41-4dce-9b7c-ce85fd8f15a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a84d026987343f4a029132a2abc61cd, user_id=7881d5d880764c43b5dbe6343de5be8a, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:12.531266 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5267d84c-cb41-4dce-9b7c-ce85fd8f15a5 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:12.542229 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1249/2482] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:25:12 2022] PUT /identity/v3/projects/3a84d026987343f4a029132a2abc61cd/users/7881d5d880764c43b5dbe6343de5be8a/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:25:12.553273 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fee8ae-a93c-400d-a1bd-78f80da9372f None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:12.553603 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fee8ae-a93c-400d-a1bd-78f80da9372f None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:12.553944 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fee8ae-a93c-400d-a1bd-78f80da9372f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:12.583434 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-16fee8ae-a93c-400d-a1bd-78f80da9372f None None] MFA Rules not processed for user `7881d5d880764c43b5dbe6343de5be8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:25:12.584637 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-16fee8ae-a93c-400d-a1bd-78f80da9372f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:12.654447 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1234/2483] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:25:12 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:25:12.665964 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5267d84c-cb41-4dce-9b7c-ce85fd8f15a5 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:12.668095 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5267d84c-cb41-4dce-9b7c-ce85fd8f15a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:12.714376 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:25:12.752249 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5267d84c-cb41-4dce-9b7c-ce85fd8f15a5 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:12.756390 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23381a78-c511-4709-bccb-efc458c9959d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:12.757025 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23381a78-c511-4709-bccb-efc458c9959d service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:12.757647 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23381a78-c511-4709-bccb-efc458c9959d service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:12.757647 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23381a78-c511-4709-bccb-efc458c9959d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:12.767922 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23381a78-c511-4709-bccb-efc458c9959d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:12.769468 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23381a78-c511-4709-bccb-efc458c9959d service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:12.779528 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1250/2484] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:25:12 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:25:12.920028 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fee8ae-a93c-400d-a1bd-78f80da9372f None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:12.985077 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fee8ae-a93c-400d-a1bd-78f80da9372f None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:12.989956 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d17317-7bbd-47c8-90b5-f99d4e71ad14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:12.990582 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d17317-7bbd-47c8-90b5-f99d4e71ad14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:12.990896 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d17317-7bbd-47c8-90b5-f99d4e71ad14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:12.991164 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d17317-7bbd-47c8-90b5-f99d4e71ad14 admin admin] PATH_INFO: `/v3/users/3493bbdeab23450c80a22389e5ac78e7` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:12.992320 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d17317-7bbd-47c8-90b5-f99d4e71ad14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3493bbdeab23450c80a22389e5ac78e7)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:12.994282 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d17317-7bbd-47c8-90b5-f99d4e71ad14 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:13.040958 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a1d17317-7bbd-47c8-90b5-f99d4e71ad14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3493bbdeab23450c80a22389e5ac78e7'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:13.041657 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a1d17317-7bbd-47c8-90b5-f99d4e71ad14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3493bbdeab23450c80a22389e5ac78e7'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:13.049390 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a1d17317-7bbd-47c8-90b5-f99d4e71ad14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3493bbdeab23450c80a22389e5ac78e7'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:13.055291 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:25:13.063993 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a1d17317-7bbd-47c8-90b5-f99d4e71ad14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3493bbdeab23450c80a22389e5ac78e7'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:13.072009 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a1d17317-7bbd-47c8-90b5-f99d4e71ad14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3493bbdeab23450c80a22389e5ac78e7'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:13.080135 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1235/2485] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:25:12 2022] DELETE /identity/v3/users/3493bbdeab23450c80a22389e5ac78e7 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:25:13.085289 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688dba01-0f9a-4328-b1e5-305b429894ee None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:13.085611 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688dba01-0f9a-4328-b1e5-305b429894ee None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:13.086005 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688dba01-0f9a-4328-b1e5-305b429894ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:13.107912 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-688dba01-0f9a-4328-b1e5-305b429894ee None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:25:13.109183 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-688dba01-0f9a-4328-b1e5-305b429894ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:13.174505 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1251/2486] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:25:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:25:13.189295 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d17317-7bbd-47c8-90b5-f99d4e71ad14 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:13.194791 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a1d17317-7bbd-47c8-90b5-f99d4e71ad14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:13.255282 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d17317-7bbd-47c8-90b5-f99d4e71ad14 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:13.259265 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc81494-506d-4816-b48d-ccf3933e7044 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:13.259868 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc81494-506d-4816-b48d-ccf3933e7044 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:13.260304 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc81494-506d-4816-b48d-ccf3933e7044 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:13.260621 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc81494-506d-4816-b48d-ccf3933e7044 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:13.270050 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc81494-506d-4816-b48d-ccf3933e7044 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:13.271534 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc81494-506d-4816-b48d-ccf3933e7044 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:13.287189 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1236/2487] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:25:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:25:13.558507 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688dba01-0f9a-4328-b1e5-305b429894ee None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:13.560023 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-688dba01-0f9a-4328-b1e5-305b429894ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:13.572198 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688dba01-0f9a-4328-b1e5-305b429894ee None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:13.577101 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2ab510-78dc-4c75-b0eb-925805204cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:13.577752 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2ab510-78dc-4c75-b0eb-925805204cab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:13.578028 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2ab510-78dc-4c75-b0eb-925805204cab admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:13.578293 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2ab510-78dc-4c75-b0eb-925805204cab admin admin] PATH_INFO: `/v3/projects/66d36e32b67442e8b754c92e12b693f0` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:13.579479 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2ab510-78dc-4c75-b0eb-925805204cab admin admin] RBAC: Authorizing `identity:delete_project(project_id=66d36e32b67442e8b754c92e12b693f0)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:13.581453 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2ab510-78dc-4c75-b0eb-925805204cab admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:13.630872 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2d2ab510-78dc-4c75-b0eb-925805204cab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66d36e32b67442e8b754c92e12b693f0', 'request_id': 'req-2d2ab510-78dc-4c75-b0eb-925805204cab'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:13.636809 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2d2ab510-78dc-4c75-b0eb-925805204cab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66d36e32b67442e8b754c92e12b693f0', 'request_id': 'req-2d2ab510-78dc-4c75-b0eb-925805204cab'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:13.645214 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2d2ab510-78dc-4c75-b0eb-925805204cab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66d36e32b67442e8b754c92e12b693f0', 'request_id': 'req-2d2ab510-78dc-4c75-b0eb-925805204cab'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:13.656505 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2d2ab510-78dc-4c75-b0eb-925805204cab admin admin] The token cache is being invalidate because project 66d36e32b67442e8b754c92e12b693f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:25:13.656815 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2d2ab510-78dc-4c75-b0eb-925805204cab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:13.657452 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2d2ab510-78dc-4c75-b0eb-925805204cab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:13.658977 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1252/2488] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:25:13 2022] DELETE /identity/v3/projects/66d36e32b67442e8b754c92e12b693f0 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:25:13.663083 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc81494-506d-4816-b48d-ccf3933e7044 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:13.673233 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc81494-506d-4816-b48d-ccf3933e7044 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:13.734744 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05df75ab-a3b8-4517-8f7d-4999f66a7b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:13.735293 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df75ab-a3b8-4517-8f7d-4999f66a7b5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:13.735594 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df75ab-a3b8-4517-8f7d-4999f66a7b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:13.735862 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df75ab-a3b8-4517-8f7d-4999f66a7b5d admin admin] PATH_INFO: `/v3/users/7881d5d880764c43b5dbe6343de5be8a` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:13.736994 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05df75ab-a3b8-4517-8f7d-4999f66a7b5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7881d5d880764c43b5dbe6343de5be8a)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:13.738909 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05df75ab-a3b8-4517-8f7d-4999f66a7b5d admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:13.786220 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-05df75ab-a3b8-4517-8f7d-4999f66a7b5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7881d5d880764c43b5dbe6343de5be8a'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:13.786880 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-05df75ab-a3b8-4517-8f7d-4999f66a7b5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7881d5d880764c43b5dbe6343de5be8a'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:13.793986 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-05df75ab-a3b8-4517-8f7d-4999f66a7b5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7881d5d880764c43b5dbe6343de5be8a'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:13.807540 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-05df75ab-a3b8-4517-8f7d-4999f66a7b5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7881d5d880764c43b5dbe6343de5be8a'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:13.816706 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-05df75ab-a3b8-4517-8f7d-4999f66a7b5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7881d5d880764c43b5dbe6343de5be8a'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:13.822854 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1237/2489] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:25:13 2022] DELETE /identity/v3/users/7881d5d880764c43b5dbe6343de5be8a => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:25:14.190323 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2ab510-78dc-4c75-b0eb-925805204cab admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:14.199209 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-2d2ab510-78dc-4c75-b0eb-925805204cab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:14.205952 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2ab510-78dc-4c75-b0eb-925805204cab admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:14.251530 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:25:14.269229 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b698aa5-fd57-432c-a1bf-c83f715d3cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:14.269873 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b698aa5-fd57-432c-a1bf-c83f715d3cff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:14.270154 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b698aa5-fd57-432c-a1bf-c83f715d3cff admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:14.270426 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b698aa5-fd57-432c-a1bf-c83f715d3cff admin admin] PATH_INFO: `/v3/projects/3a84d026987343f4a029132a2abc61cd` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:14.271576 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b698aa5-fd57-432c-a1bf-c83f715d3cff admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a84d026987343f4a029132a2abc61cd)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:14.273611 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b698aa5-fd57-432c-a1bf-c83f715d3cff admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:14.321385 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2b698aa5-fd57-432c-a1bf-c83f715d3cff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a84d026987343f4a029132a2abc61cd', 'request_id': 'req-2b698aa5-fd57-432c-a1bf-c83f715d3cff'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:14.328363 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2b698aa5-fd57-432c-a1bf-c83f715d3cff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a84d026987343f4a029132a2abc61cd', 'request_id': 'req-2b698aa5-fd57-432c-a1bf-c83f715d3cff'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:14.338126 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2b698aa5-fd57-432c-a1bf-c83f715d3cff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a84d026987343f4a029132a2abc61cd', 'request_id': 'req-2b698aa5-fd57-432c-a1bf-c83f715d3cff'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:14.349458 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2b698aa5-fd57-432c-a1bf-c83f715d3cff admin admin] The token cache is being invalidate because project 3a84d026987343f4a029132a2abc61cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:25:14.349743 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2b698aa5-fd57-432c-a1bf-c83f715d3cff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:14.350435 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2b698aa5-fd57-432c-a1bf-c83f715d3cff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:14.351933 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1253/2490] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:25:14 2022] DELETE /identity/v3/projects/3a84d026987343f4a029132a2abc61cd => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:25:14.357395 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8c6136-74b0-4499-b926-9aa5dfe48d83 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:14.357868 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8c6136-74b0-4499-b926-9aa5dfe48d83 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:14.357986 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8c6136-74b0-4499-b926-9aa5dfe48d83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:14.365830 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:25:14.379835 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-8d8c6136-74b0-4499-b926-9aa5dfe48d83 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:25:14.381083 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8d8c6136-74b0-4499-b926-9aa5dfe48d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:14.440570 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1238/2491] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:25:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:25:14.449837 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9782aa85-863d-475b-a49d-bcd7d0414ecb None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:14.450115 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9782aa85-863d-475b-a49d-bcd7d0414ecb None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:14.450386 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9782aa85-863d-475b-a49d-bcd7d0414ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:14.471243 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-9782aa85-863d-475b-a49d-bcd7d0414ecb None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:25:14.472444 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-9782aa85-863d-475b-a49d-bcd7d0414ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:14.480660 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1254/2492] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:25:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:25:14.485198 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8c6136-74b0-4499-b926-9aa5dfe48d83 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:14.495486 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8c6136-74b0-4499-b926-9aa5dfe48d83 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:14.501290 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6a4ec2-2c1a-4619-9494-916a794fa8a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:14.501888 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6a4ec2-2c1a-4619-9494-916a794fa8a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:14.502153 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6a4ec2-2c1a-4619-9494-916a794fa8a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:14.502409 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6a4ec2-2c1a-4619-9494-916a794fa8a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:14.503052 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6a4ec2-2c1a-4619-9494-916a794fa8a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:25:14.503346 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6a4ec2-2c1a-4619-9494-916a794fa8a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:14.504978 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6a4ec2-2c1a-4619-9494-916a794fa8a6 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:14.516809 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1239/2493] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 6 16:25:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:25:14.521401 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9782aa85-863d-475b-a49d-bcd7d0414ecb None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:14.531699 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9782aa85-863d-475b-a49d-bcd7d0414ecb None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:14.537403 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bf84b8-285e-4722-ab21-46a49a41e45d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:14.537990 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf84b8-285e-4722-ab21-46a49a41e45d admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:14.538266 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf84b8-285e-4722-ab21-46a49a41e45d admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:14.538534 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf84b8-285e-4722-ab21-46a49a41e45d admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:14.539196 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bf84b8-285e-4722-ab21-46a49a41e45d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:14.541167 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bf84b8-285e-4722-ab21-46a49a41e45d admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:14.569568 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1255/2494] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:25:14 2022] POST /identity/v3/projects => generated 409 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:25:14.573720 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6a4ec2-2c1a-4619-9494-916a794fa8a6 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:14.583863 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6a4ec2-2c1a-4619-9494-916a794fa8a6 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:14.642653 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5d3196-3fb2-404c-bffa-763ee563675d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:14.643283 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5d3196-3fb2-404c-bffa-763ee563675d admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:14.643555 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5d3196-3fb2-404c-bffa-763ee563675d admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:14.643823 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5d3196-3fb2-404c-bffa-763ee563675d admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:14.644464 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5d3196-3fb2-404c-bffa-763ee563675d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:14.646445 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5d3196-3fb2-404c-bffa-763ee563675d admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:14.668055 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1240/2495] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:25:14 2022] POST /identity/v3/users => generated 305 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:25:14.672337 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bf84b8-285e-4722-ab21-46a49a41e45d admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:14.683241 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bf84b8-285e-4722-ab21-46a49a41e45d admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:14.688893 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aeb61a-4b07-461b-91fe-d9be2f667d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:14.689496 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aeb61a-4b07-461b-91fe-d9be2f667d7d admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:14.689772 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aeb61a-4b07-461b-91fe-d9be2f667d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:14.690038 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aeb61a-4b07-461b-91fe-d9be2f667d7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:14.690681 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aeb61a-4b07-461b-91fe-d9be2f667d7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:14.692414 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aeb61a-4b07-461b-91fe-d9be2f667d7d admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:14.703103 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1256/2496] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:25:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:25:14.706987 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5d3196-3fb2-404c-bffa-763ee563675d admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:14.717729 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5d3196-3fb2-404c-bffa-763ee563675d admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:14.723206 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6182b9-7079-4fdc-910d-fe43f10c1d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:14.723816 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6182b9-7079-4fdc-910d-fe43f10c1d4c admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:14.724141 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6182b9-7079-4fdc-910d-fe43f10c1d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:14.724396 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6182b9-7079-4fdc-910d-fe43f10c1d4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:14.725181 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6182b9-7079-4fdc-910d-fe43f10c1d4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:14.726840 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6182b9-7079-4fdc-910d-fe43f10c1d4c admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:14.738025 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1241/2497] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:25:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:25:14.742522 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aeb61a-4b07-461b-91fe-d9be2f667d7d admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:14.752935 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aeb61a-4b07-461b-91fe-d9be2f667d7d admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:14.758331 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f83ee40-51e6-473b-9ead-3951e48b104f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:14.759480 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f83ee40-51e6-473b-9ead-3951e48b104f admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:14.759755 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f83ee40-51e6-473b-9ead-3951e48b104f admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:14.760027 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f83ee40-51e6-473b-9ead-3951e48b104f admin admin] PATH_INFO: `/v3/projects/7a4f9d3ed590418e96086e9c606e94df/users/7b0d5c5e43f645919503174104b98451/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:14.775968 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f83ee40-51e6-473b-9ead-3951e48b104f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a4f9d3ed590418e96086e9c606e94df, user_id=7b0d5c5e43f645919503174104b98451, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:14.778644 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f83ee40-51e6-473b-9ead-3951e48b104f admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:14.792695 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1257/2498] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:25:14 2022] PUT /identity/v3/projects/7a4f9d3ed590418e96086e9c606e94df/users/7b0d5c5e43f645919503174104b98451/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:25:14.803550 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0443892c-82aa-4d88-8f77-08521022f26c None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:14.803829 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0443892c-82aa-4d88-8f77-08521022f26c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:14.804098 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0443892c-82aa-4d88-8f77-08521022f26c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:14.832608 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-0443892c-82aa-4d88-8f77-08521022f26c None None] MFA Rules not processed for user `7b0d5c5e43f645919503174104b98451`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:25:14.833840 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0443892c-82aa-4d88-8f77-08521022f26c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:14.896879 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:25:14.903398 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1242/2499] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:25:14 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:25:14.908213 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f83ee40-51e6-473b-9ead-3951e48b104f admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:14.971583 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f83ee40-51e6-473b-9ead-3951e48b104f admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:14.977235 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97f355a-e503-4ae8-a090-355b7d40b5fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:14.977840 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97f355a-e503-4ae8-a090-355b7d40b5fa admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:14.978116 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97f355a-e503-4ae8-a090-355b7d40b5fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:14.978419 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97f355a-e503-4ae8-a090-355b7d40b5fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:14.979035 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97f355a-e503-4ae8-a090-355b7d40b5fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:14.981065 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97f355a-e503-4ae8-a090-355b7d40b5fa admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:14.996657 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:25:15.011580 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1258/2500] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:25:14 2022] POST /identity/v3/projects => generated 407 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:25:15.015905 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0443892c-82aa-4d88-8f77-08521022f26c None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:15.029750 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0443892c-82aa-4d88-8f77-08521022f26c None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:15.089950 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e5e2ab-1fe9-4e5e-b5a9-b5806094ef53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:15.090539 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e5e2ab-1fe9-4e5e-b5a9-b5806094ef53 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:15.090810 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e5e2ab-1fe9-4e5e-b5a9-b5806094ef53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:15.091075 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e5e2ab-1fe9-4e5e-b5a9-b5806094ef53 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:15.091708 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e5e2ab-1fe9-4e5e-b5a9-b5806094ef53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:15.093651 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e5e2ab-1fe9-4e5e-b5a9-b5806094ef53 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:15.114198 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1243/2501] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:25:15 2022] POST /identity/v3/users => generated 310 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:25:15.119313 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97f355a-e503-4ae8-a090-355b7d40b5fa admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:15.129667 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97f355a-e503-4ae8-a090-355b7d40b5fa admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:15.134615 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cc4ee4-471e-4b27-b361-a78fb9e3703d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:15.135242 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cc4ee4-471e-4b27-b361-a78fb9e3703d admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:15.135513 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cc4ee4-471e-4b27-b361-a78fb9e3703d admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:15.135776 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cc4ee4-471e-4b27-b361-a78fb9e3703d admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:15.136409 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cc4ee4-471e-4b27-b361-a78fb9e3703d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:15.138232 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cc4ee4-471e-4b27-b361-a78fb9e3703d admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:15.149068 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1259/2502] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:25:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:25:15.153568 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e5e2ab-1fe9-4e5e-b5a9-b5806094ef53 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:15.164080 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e5e2ab-1fe9-4e5e-b5a9-b5806094ef53 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:15.169655 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2487a3-0337-4370-876f-a361facf291a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:15.170323 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2487a3-0337-4370-876f-a361facf291a admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:15.170577 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2487a3-0337-4370-876f-a361facf291a admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:15.170845 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2487a3-0337-4370-876f-a361facf291a admin admin] PATH_INFO: `/v3/projects/0ee8e4b613e647a7aec80c5c4af44729/users/ef8048132e6d4256b490d7afae3c3042/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:15.187315 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2487a3-0337-4370-876f-a361facf291a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ee8e4b613e647a7aec80c5c4af44729, user_id=ef8048132e6d4256b490d7afae3c3042, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:15.190035 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2487a3-0337-4370-876f-a361facf291a admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:15.205704 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1244/2503] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:25:15 2022] PUT /identity/v3/projects/0ee8e4b613e647a7aec80c5c4af44729/users/ef8048132e6d4256b490d7afae3c3042/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:25:15.209780 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cc4ee4-471e-4b27-b361-a78fb9e3703d admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:15.220779 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cc4ee4-471e-4b27-b361-a78fb9e3703d admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:15.282222 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee7151a-186b-4d61-ad4a-ecc560ad006e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:15.282988 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee7151a-186b-4d61-ad4a-ecc560ad006e admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:15.283258 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee7151a-186b-4d61-ad4a-ecc560ad006e admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:15.283525 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee7151a-186b-4d61-ad4a-ecc560ad006e admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:15.284173 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee7151a-186b-4d61-ad4a-ecc560ad006e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:15.285936 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee7151a-186b-4d61-ad4a-ecc560ad006e admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:15.296882 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1260/2504] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:25:15 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:25:15.300890 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2487a3-0337-4370-876f-a361facf291a admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:15.312280 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2487a3-0337-4370-876f-a361facf291a admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:15.317258 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2921c5-63a7-46e2-8748-44bcbfa9b29b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:15.317914 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2921c5-63a7-46e2-8748-44bcbfa9b29b admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:15.318190 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2921c5-63a7-46e2-8748-44bcbfa9b29b admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:15.318461 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2921c5-63a7-46e2-8748-44bcbfa9b29b admin admin] PATH_INFO: `/v3/projects/0ee8e4b613e647a7aec80c5c4af44729/users/ef8048132e6d4256b490d7afae3c3042/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:15.321950 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2921c5-63a7-46e2-8748-44bcbfa9b29b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ee8e4b613e647a7aec80c5c4af44729, user_id=ef8048132e6d4256b490d7afae3c3042, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:15.324582 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2921c5-63a7-46e2-8748-44bcbfa9b29b admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:15.338180 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1245/2505] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:25:15 2022] PUT /identity/v3/projects/0ee8e4b613e647a7aec80c5c4af44729/users/ef8048132e6d4256b490d7afae3c3042/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:25:15.348410 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c33fc6-5c9c-4b71-9bf2-03acaae9c8e7 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:15.348670 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c33fc6-5c9c-4b71-9bf2-03acaae9c8e7 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:15.348965 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c33fc6-5c9c-4b71-9bf2-03acaae9c8e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:15.380596 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-d2c33fc6-5c9c-4b71-9bf2-03acaae9c8e7 None None] MFA Rules not processed for user `ef8048132e6d4256b490d7afae3c3042`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:25:15.381912 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d2c33fc6-5c9c-4b71-9bf2-03acaae9c8e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:15.448836 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1261/2506] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:25:15 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:25:15.460260 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2921c5-63a7-46e2-8748-44bcbfa9b29b admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:15.462222 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ce2921c5-63a7-46e2-8748-44bcbfa9b29b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:15.521899 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2921c5-63a7-46e2-8748-44bcbfa9b29b admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:15.526314 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cc7190-6df3-4b09-acb7-c9e69decf5a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:15.526921 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cc7190-6df3-4b09-acb7-c9e69decf5a6 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:15.527200 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cc7190-6df3-4b09-acb7-c9e69decf5a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:15.527460 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cc7190-6df3-4b09-acb7-c9e69decf5a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:15.538052 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cc7190-6df3-4b09-acb7-c9e69decf5a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:15.539512 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cc7190-6df3-4b09-acb7-c9e69decf5a6 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:15.549229 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1246/2507] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:25:15 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:25:15.710401 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c33fc6-5c9c-4b71-9bf2-03acaae9c8e7 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:15.720894 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c33fc6-5c9c-4b71-9bf2-03acaae9c8e7 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:15.725403 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0275032-3f74-47d8-b0d5-85f8d4955477 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:15.726020 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0275032-3f74-47d8-b0d5-85f8d4955477 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:15.726278 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0275032-3f74-47d8-b0d5-85f8d4955477 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:15.726546 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0275032-3f74-47d8-b0d5-85f8d4955477 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:15.786356 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0275032-3f74-47d8-b0d5-85f8d4955477 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:15.787801 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0275032-3f74-47d8-b0d5-85f8d4955477 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:15.797125 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1262/2508] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:25:15 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:25:25.319435 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cc7190-6df3-4b09-acb7-c9e69decf5a6 service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:25.379859 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:25:25.393164 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cc7190-6df3-4b09-acb7-c9e69decf5a6 service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:25.398229 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cf08cb-2a53-43b9-b25c-ac68ec6f73a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:25.398856 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cf08cb-2a53-43b9-b25c-ac68ec6f73a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:25.399130 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cf08cb-2a53-43b9-b25c-ac68ec6f73a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:25.399398 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cf08cb-2a53-43b9-b25c-ac68ec6f73a3 admin admin] PATH_INFO: `/v3/users/7b0d5c5e43f645919503174104b98451` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:25.400600 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cf08cb-2a53-43b9-b25c-ac68ec6f73a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b0d5c5e43f645919503174104b98451)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:25.402572 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cf08cb-2a53-43b9-b25c-ac68ec6f73a3 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:25.451404 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-29cf08cb-2a53-43b9-b25c-ac68ec6f73a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b0d5c5e43f645919503174104b98451'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:25.452071 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-29cf08cb-2a53-43b9-b25c-ac68ec6f73a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b0d5c5e43f645919503174104b98451'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:25.458996 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-29cf08cb-2a53-43b9-b25c-ac68ec6f73a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b0d5c5e43f645919503174104b98451'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:25.470954 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-29cf08cb-2a53-43b9-b25c-ac68ec6f73a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b0d5c5e43f645919503174104b98451'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:25.479950 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-29cf08cb-2a53-43b9-b25c-ac68ec6f73a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b0d5c5e43f645919503174104b98451'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:25.485968 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1247/2509] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:25:25 2022] DELETE /identity/v3/users/7b0d5c5e43f645919503174104b98451 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:25:25.490785 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2126f073-7df8-4c36-990c-d1e380a15265 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:25.491597 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2126f073-7df8-4c36-990c-d1e380a15265 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:25.491873 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2126f073-7df8-4c36-990c-d1e380a15265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:25.500622 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:25:25.512677 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-2126f073-7df8-4c36-990c-d1e380a15265 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:25:25.513932 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-2126f073-7df8-4c36-990c-d1e380a15265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:25.573863 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1263/2510] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:25:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:25:25.585768 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cf08cb-2a53-43b9-b25c-ac68ec6f73a3 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:25.587422 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-29cf08cb-2a53-43b9-b25c-ac68ec6f73a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:25.652414 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cf08cb-2a53-43b9-b25c-ac68ec6f73a3 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:25.656622 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389441bf-861f-4c6e-9df2-05788dc4dea1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:25.657296 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389441bf-861f-4c6e-9df2-05788dc4dea1 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:25.657572 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389441bf-861f-4c6e-9df2-05788dc4dea1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:25.657838 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389441bf-861f-4c6e-9df2-05788dc4dea1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:25.669233 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389441bf-861f-4c6e-9df2-05788dc4dea1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:25.670677 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389441bf-861f-4c6e-9df2-05788dc4dea1 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:25.680561 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1248/2511] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:25:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:25:25.959251 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2126f073-7df8-4c36-990c-d1e380a15265 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:25.960778 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-2126f073-7df8-4c36-990c-d1e380a15265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:25.977863 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2126f073-7df8-4c36-990c-d1e380a15265 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:25.983704 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37a08f5-3f23-49da-ae0c-0bfefc5a81f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:25.984447 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37a08f5-3f23-49da-ae0c-0bfefc5a81f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:25.984637 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37a08f5-3f23-49da-ae0c-0bfefc5a81f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:25.985025 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37a08f5-3f23-49da-ae0c-0bfefc5a81f7 admin admin] PATH_INFO: `/v3/projects/7a4f9d3ed590418e96086e9c606e94df` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:25.986305 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37a08f5-3f23-49da-ae0c-0bfefc5a81f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a4f9d3ed590418e96086e9c606e94df)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:25.988239 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37a08f5-3f23-49da-ae0c-0bfefc5a81f7 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:26.044158 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e37a08f5-3f23-49da-ae0c-0bfefc5a81f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a4f9d3ed590418e96086e9c606e94df', 'request_id': 'req-e37a08f5-3f23-49da-ae0c-0bfefc5a81f7'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:26.050201 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e37a08f5-3f23-49da-ae0c-0bfefc5a81f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a4f9d3ed590418e96086e9c606e94df', 'request_id': 'req-e37a08f5-3f23-49da-ae0c-0bfefc5a81f7'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:26.059190 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e37a08f5-3f23-49da-ae0c-0bfefc5a81f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a4f9d3ed590418e96086e9c606e94df', 'request_id': 'req-e37a08f5-3f23-49da-ae0c-0bfefc5a81f7'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:26.071549 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e37a08f5-3f23-49da-ae0c-0bfefc5a81f7 admin admin] The token cache is being invalidate because project 7a4f9d3ed590418e96086e9c606e94df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:25:26.071818 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e37a08f5-3f23-49da-ae0c-0bfefc5a81f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:26.072971 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e37a08f5-3f23-49da-ae0c-0bfefc5a81f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:26.073973 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1264/2512] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:25:25 2022] DELETE /identity/v3/projects/7a4f9d3ed590418e96086e9c606e94df => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:25:26.077978 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389441bf-861f-4c6e-9df2-05788dc4dea1 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:26.087955 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389441bf-861f-4c6e-9df2-05788dc4dea1 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:26.145806 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2af020-a4e6-4692-bc9b-526e78fa65ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:26.146552 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2af020-a4e6-4692-bc9b-526e78fa65ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:26.146826 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2af020-a4e6-4692-bc9b-526e78fa65ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:26.147092 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2af020-a4e6-4692-bc9b-526e78fa65ed admin admin] PATH_INFO: `/v3/users/ef8048132e6d4256b490d7afae3c3042` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:26.148202 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2af020-a4e6-4692-bc9b-526e78fa65ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef8048132e6d4256b490d7afae3c3042)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:26.150142 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2af020-a4e6-4692-bc9b-526e78fa65ed admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:26.165949 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:25:26.198735 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-dd2af020-a4e6-4692-bc9b-526e78fa65ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef8048132e6d4256b490d7afae3c3042'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:26.199387 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-dd2af020-a4e6-4692-bc9b-526e78fa65ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef8048132e6d4256b490d7afae3c3042'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:26.213311 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-dd2af020-a4e6-4692-bc9b-526e78fa65ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef8048132e6d4256b490d7afae3c3042'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:26.228012 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-dd2af020-a4e6-4692-bc9b-526e78fa65ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef8048132e6d4256b490d7afae3c3042'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:26.240056 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-dd2af020-a4e6-4692-bc9b-526e78fa65ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef8048132e6d4256b490d7afae3c3042'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:26.250104 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1249/2513] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:25:26 2022] DELETE /identity/v3/users/ef8048132e6d4256b490d7afae3c3042 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:25:26.526216 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37a08f5-3f23-49da-ae0c-0bfefc5a81f7 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:26.527791 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e37a08f5-3f23-49da-ae0c-0bfefc5a81f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:26.544109 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37a08f5-3f23-49da-ae0c-0bfefc5a81f7 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:26.606894 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb19e7a-a14a-4a06-a67c-9a8fab8bc6b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:26.607531 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb19e7a-a14a-4a06-a67c-9a8fab8bc6b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:26.607802 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb19e7a-a14a-4a06-a67c-9a8fab8bc6b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:26.608073 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb19e7a-a14a-4a06-a67c-9a8fab8bc6b2 admin admin] PATH_INFO: `/v3/projects/0ee8e4b613e647a7aec80c5c4af44729` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:26.609266 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb19e7a-a14a-4a06-a67c-9a8fab8bc6b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ee8e4b613e647a7aec80c5c4af44729)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:26.611229 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb19e7a-a14a-4a06-a67c-9a8fab8bc6b2 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:26.647349 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:25:26.661746 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-0eb19e7a-a14a-4a06-a67c-9a8fab8bc6b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ee8e4b613e647a7aec80c5c4af44729', 'request_id': 'req-0eb19e7a-a14a-4a06-a67c-9a8fab8bc6b2'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:26.667240 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-0eb19e7a-a14a-4a06-a67c-9a8fab8bc6b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ee8e4b613e647a7aec80c5c4af44729', 'request_id': 'req-0eb19e7a-a14a-4a06-a67c-9a8fab8bc6b2'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:26.675114 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-0eb19e7a-a14a-4a06-a67c-9a8fab8bc6b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ee8e4b613e647a7aec80c5c4af44729', 'request_id': 'req-0eb19e7a-a14a-4a06-a67c-9a8fab8bc6b2'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:26.687058 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-0eb19e7a-a14a-4a06-a67c-9a8fab8bc6b2 admin admin] The token cache is being invalidate because project 0ee8e4b613e647a7aec80c5c4af44729 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:25:26.687352 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-0eb19e7a-a14a-4a06-a67c-9a8fab8bc6b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:26.688004 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-0eb19e7a-a14a-4a06-a67c-9a8fab8bc6b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:26.689555 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1265/2514] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:25:26 2022] DELETE /identity/v3/projects/0ee8e4b613e647a7aec80c5c4af44729 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:25:26.694534 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68470ff9-77fd-4c3a-ba92-84bfd48c9cd4 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:26.695030 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68470ff9-77fd-4c3a-ba92-84bfd48c9cd4 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:26.695030 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68470ff9-77fd-4c3a-ba92-84bfd48c9cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:26.716612 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-68470ff9-77fd-4c3a-ba92-84bfd48c9cd4 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:25:26.717760 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-68470ff9-77fd-4c3a-ba92-84bfd48c9cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:26.784185 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1250/2515] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:25:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:25:26.793775 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5971d24f-73d1-43a5-8d08-326924f9b154 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:26.794127 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5971d24f-73d1-43a5-8d08-326924f9b154 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:26.794405 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5971d24f-73d1-43a5-8d08-326924f9b154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:26.816800 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-5971d24f-73d1-43a5-8d08-326924f9b154 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:25:26.818122 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5971d24f-73d1-43a5-8d08-326924f9b154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:26.826816 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1266/2516] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:25:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:25:26.831790 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68470ff9-77fd-4c3a-ba92-84bfd48c9cd4 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:26.842321 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68470ff9-77fd-4c3a-ba92-84bfd48c9cd4 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:26.849784 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdc488e-72c2-466d-a749-4d6be5532649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:26.850679 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdc488e-72c2-466d-a749-4d6be5532649 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:26.851088 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdc488e-72c2-466d-a749-4d6be5532649 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:26.851440 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdc488e-72c2-466d-a749-4d6be5532649 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:26.852431 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdc488e-72c2-466d-a749-4d6be5532649 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:25:26.852936 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdc488e-72c2-466d-a749-4d6be5532649 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:26.855261 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdc488e-72c2-466d-a749-4d6be5532649 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:26.867797 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1251/2517] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 6 16:25:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:25:26.872472 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5971d24f-73d1-43a5-8d08-326924f9b154 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:26.881966 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5971d24f-73d1-43a5-8d08-326924f9b154 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:26.887802 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1faf2f9-81fd-42ee-ac1e-1038206a644c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:26.888424 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1faf2f9-81fd-42ee-ac1e-1038206a644c admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:26.888725 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1faf2f9-81fd-42ee-ac1e-1038206a644c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:26.889009 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1faf2f9-81fd-42ee-ac1e-1038206a644c admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:26.889648 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1faf2f9-81fd-42ee-ac1e-1038206a644c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:26.891571 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1faf2f9-81fd-42ee-ac1e-1038206a644c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:26.920950 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1267/2518] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:25:26 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:25:26.925373 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdc488e-72c2-466d-a749-4d6be5532649 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:26.936084 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdc488e-72c2-466d-a749-4d6be5532649 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:26.996871 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fb99bb-8149-4657-8ab8-844c02a1dc6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:26.997457 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb99bb-8149-4657-8ab8-844c02a1dc6a admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:26.997732 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb99bb-8149-4657-8ab8-844c02a1dc6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:26.998004 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb99bb-8149-4657-8ab8-844c02a1dc6a admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:26.998630 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fb99bb-8149-4657-8ab8-844c02a1dc6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:27.000600 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fb99bb-8149-4657-8ab8-844c02a1dc6a admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:27.027523 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1252/2519] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:25:26 2022] POST /identity/v3/users => generated 286 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:25:27.032560 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1faf2f9-81fd-42ee-ac1e-1038206a644c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:27.047464 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1faf2f9-81fd-42ee-ac1e-1038206a644c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:27.055614 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaf1461-8e4a-49cd-8b75-849e0a65c735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:27.056524 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaf1461-8e4a-49cd-8b75-849e0a65c735 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:27.056977 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaf1461-8e4a-49cd-8b75-849e0a65c735 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:27.057358 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaf1461-8e4a-49cd-8b75-849e0a65c735 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:27.058315 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaf1461-8e4a-49cd-8b75-849e0a65c735 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:27.060776 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaf1461-8e4a-49cd-8b75-849e0a65c735 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:27.071503 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1268/2520] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:25:27 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:25:27.077396 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fb99bb-8149-4657-8ab8-844c02a1dc6a admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:27.087851 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fb99bb-8149-4657-8ab8-844c02a1dc6a admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:27.093828 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36328d8b-939c-4832-afb0-f8a548d91bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:27.094433 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36328d8b-939c-4832-afb0-f8a548d91bfc admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:27.094694 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36328d8b-939c-4832-afb0-f8a548d91bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:27.094963 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36328d8b-939c-4832-afb0-f8a548d91bfc admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:27.095605 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36328d8b-939c-4832-afb0-f8a548d91bfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:27.097303 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36328d8b-939c-4832-afb0-f8a548d91bfc admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:27.102850 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:25:27.108814 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1253/2521] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:25:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:25:27.113282 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaf1461-8e4a-49cd-8b75-849e0a65c735 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:27.124506 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaf1461-8e4a-49cd-8b75-849e0a65c735 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:27.129900 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cbaaff-1174-46cb-865e-720bf6260912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:27.130636 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cbaaff-1174-46cb-865e-720bf6260912 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:27.130902 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cbaaff-1174-46cb-865e-720bf6260912 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:27.131288 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cbaaff-1174-46cb-865e-720bf6260912 admin admin] PATH_INFO: `/v3/projects/ddade75696454ef7bd7794c4e70370fb/users/2c0df058d6c946b88186c35d09a17b78/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:27.146946 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cbaaff-1174-46cb-865e-720bf6260912 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddade75696454ef7bd7794c4e70370fb, user_id=2c0df058d6c946b88186c35d09a17b78, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:27.149669 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cbaaff-1174-46cb-865e-720bf6260912 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:27.164164 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1269/2522] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:25:27 2022] PUT /identity/v3/projects/ddade75696454ef7bd7794c4e70370fb/users/2c0df058d6c946b88186c35d09a17b78/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:25:27.169134 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b949f59-06a5-44e7-8e54-1c7176a075d5 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:27.169484 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b949f59-06a5-44e7-8e54-1c7176a075d5 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:27.169820 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b949f59-06a5-44e7-8e54-1c7176a075d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:27.189778 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-1b949f59-06a5-44e7-8e54-1c7176a075d5 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:25:27.190975 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-1b949f59-06a5-44e7-8e54-1c7176a075d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:27.254716 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1254/2523] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:25:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:25:27.264783 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cbaaff-1174-46cb-865e-720bf6260912 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:27.266616 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e6cbaaff-1174-46cb-865e-720bf6260912 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:27.329099 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cbaaff-1174-46cb-865e-720bf6260912 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:27.332998 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5f244c-269c-4c89-be46-0a71ae425427 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:27.333603 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f244c-269c-4c89-be46-0a71ae425427 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:27.333879 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f244c-269c-4c89-be46-0a71ae425427 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:27.334148 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f244c-269c-4c89-be46-0a71ae425427 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:27.343302 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5f244c-269c-4c89-be46-0a71ae425427 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:27.344747 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5f244c-269c-4c89-be46-0a71ae425427 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:27.354168 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1270/2524] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:25:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:25:28.087277 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41560ea9-57aa-43a6-803b-893552c7d67a None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:28.087798 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41560ea9-57aa-43a6-803b-893552c7d67a None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:28.087798 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41560ea9-57aa-43a6-803b-893552c7d67a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:28.116391 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-41560ea9-57aa-43a6-803b-893552c7d67a None None] MFA Rules not processed for user `2c0df058d6c946b88186c35d09a17b78`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:25:28.117634 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-41560ea9-57aa-43a6-803b-893552c7d67a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:28.186787 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1255/2525] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:25:28 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:25:28.191362 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5f244c-269c-4c89-be46-0a71ae425427 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:28.201521 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5f244c-269c-4c89-be46-0a71ae425427 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:28.212315 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25d89f3-d0b9-4899-b0d3-fad832a7d113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:28.212973 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25d89f3-d0b9-4899-b0d3-fad832a7d113 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:28.213250 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25d89f3-d0b9-4899-b0d3-fad832a7d113 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:28.213518 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25d89f3-d0b9-4899-b0d3-fad832a7d113 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:28.214179 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25d89f3-d0b9-4899-b0d3-fad832a7d113 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:28.216204 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25d89f3-d0b9-4899-b0d3-fad832a7d113 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:28.244083 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1271/2526] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:25:28 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:25:28.248123 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41560ea9-57aa-43a6-803b-893552c7d67a None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:28.259066 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41560ea9-57aa-43a6-803b-893552c7d67a None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:28.327669 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ed7530-eccd-462a-aa39-fa147dd54225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:28.328465 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ed7530-eccd-462a-aa39-fa147dd54225 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:28.328872 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ed7530-eccd-462a-aa39-fa147dd54225 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:28.329259 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ed7530-eccd-462a-aa39-fa147dd54225 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:28.330058 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ed7530-eccd-462a-aa39-fa147dd54225 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:28.332324 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ed7530-eccd-462a-aa39-fa147dd54225 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:28.361520 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1256/2527] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:25:28 2022] POST /identity/v3/users => generated 293 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:25:28.363727 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25d89f3-d0b9-4899-b0d3-fad832a7d113 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:28.374873 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25d89f3-d0b9-4899-b0d3-fad832a7d113 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:28.380441 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fedcd5-a9f3-4b43-ad64-63b2a3b8682c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:28.381808 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fedcd5-a9f3-4b43-ad64-63b2a3b8682c admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:28.382087 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fedcd5-a9f3-4b43-ad64-63b2a3b8682c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:28.382353 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fedcd5-a9f3-4b43-ad64-63b2a3b8682c admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:28.383169 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fedcd5-a9f3-4b43-ad64-63b2a3b8682c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:28.385422 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fedcd5-a9f3-4b43-ad64-63b2a3b8682c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:28.398781 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1272/2528] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:25:28 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:25:28.402780 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ed7530-eccd-462a-aa39-fa147dd54225 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:28.414370 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ed7530-eccd-462a-aa39-fa147dd54225 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:28.419585 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5eaa25-8ee2-42f2-8941-bd92f5dc7662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:28.420255 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5eaa25-8ee2-42f2-8941-bd92f5dc7662 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:28.420528 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5eaa25-8ee2-42f2-8941-bd92f5dc7662 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:28.420840 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5eaa25-8ee2-42f2-8941-bd92f5dc7662 admin admin] PATH_INFO: `/v3/projects/46f1d0af2b1d434886dfb02672df9681/users/978a56bc4be040a997f17214161e15f5/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:28.439665 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5eaa25-8ee2-42f2-8941-bd92f5dc7662 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46f1d0af2b1d434886dfb02672df9681, user_id=978a56bc4be040a997f17214161e15f5, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:28.442372 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5eaa25-8ee2-42f2-8941-bd92f5dc7662 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:28.447160 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:25:28.462939 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1257/2529] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:25:28 2022] PUT /identity/v3/projects/46f1d0af2b1d434886dfb02672df9681/users/978a56bc4be040a997f17214161e15f5/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:25:28.466999 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fedcd5-a9f3-4b43-ad64-63b2a3b8682c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:28.488190 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fedcd5-a9f3-4b43-ad64-63b2a3b8682c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:28.508842 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:25:28.548647 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c701db4-e8e0-4cea-8ab9-8d77a1c9e0fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:28.549242 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c701db4-e8e0-4cea-8ab9-8d77a1c9e0fd admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:28.549521 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c701db4-e8e0-4cea-8ab9-8d77a1c9e0fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:28.549794 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c701db4-e8e0-4cea-8ab9-8d77a1c9e0fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:28.550443 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c701db4-e8e0-4cea-8ab9-8d77a1c9e0fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:28.552115 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c701db4-e8e0-4cea-8ab9-8d77a1c9e0fd admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:28.562806 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1273/2530] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:25:28 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:25:28.567426 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5eaa25-8ee2-42f2-8941-bd92f5dc7662 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:28.578416 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5eaa25-8ee2-42f2-8941-bd92f5dc7662 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:28.583637 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d65187b-a60e-464e-a90f-34672d2afe74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:28.584305 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d65187b-a60e-464e-a90f-34672d2afe74 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:28.584577 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d65187b-a60e-464e-a90f-34672d2afe74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:28.584889 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d65187b-a60e-464e-a90f-34672d2afe74 admin admin] PATH_INFO: `/v3/projects/46f1d0af2b1d434886dfb02672df9681/users/978a56bc4be040a997f17214161e15f5/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:28.586931 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d65187b-a60e-464e-a90f-34672d2afe74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46f1d0af2b1d434886dfb02672df9681, user_id=978a56bc4be040a997f17214161e15f5, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:28.589637 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d65187b-a60e-464e-a90f-34672d2afe74 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:28.601389 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1258/2531] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:25:28 2022] PUT /identity/v3/projects/46f1d0af2b1d434886dfb02672df9681/users/978a56bc4be040a997f17214161e15f5/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:25:29.561246 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7c2138-47a3-4113-9538-20c04a160af0 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:29.561813 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7c2138-47a3-4113-9538-20c04a160af0 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:29.561813 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7c2138-47a3-4113-9538-20c04a160af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:29.592732 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-1f7c2138-47a3-4113-9538-20c04a160af0 None None] MFA Rules not processed for user `978a56bc4be040a997f17214161e15f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:25:29.593362 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1f7c2138-47a3-4113-9538-20c04a160af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:29.684015 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1274/2532] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:25:29 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:25:29.698018 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d65187b-a60e-464e-a90f-34672d2afe74 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:29.699840 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7d65187b-a60e-464e-a90f-34672d2afe74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:29.762969 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d65187b-a60e-464e-a90f-34672d2afe74 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:29.766820 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f11caa-5ae4-4a73-8982-933159c46770 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:29.767435 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f11caa-5ae4-4a73-8982-933159c46770 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:29.767705 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f11caa-5ae4-4a73-8982-933159c46770 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:29.767966 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f11caa-5ae4-4a73-8982-933159c46770 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:29.824292 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f11caa-5ae4-4a73-8982-933159c46770 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:29.825754 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f11caa-5ae4-4a73-8982-933159c46770 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:29.836845 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1259/2533] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:25:29 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:25:31.105427 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0c7116-9f0a-4dfe-833c-993e3e7301b2 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:31.105976 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0c7116-9f0a-4dfe-833c-993e3e7301b2 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:31.105976 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0c7116-9f0a-4dfe-833c-993e3e7301b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:31.125949 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-bf0c7116-9f0a-4dfe-833c-993e3e7301b2 None None] MFA Rules not processed for user `4d3856b7bb7f4f5bb801952b893bbe35`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:25:31.127147 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-bf0c7116-9f0a-4dfe-833c-993e3e7301b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:31.184349 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1275/2534] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:25:31 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:25:31.289032 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f11caa-5ae4-4a73-8982-933159c46770 service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:31.298445 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f11caa-5ae4-4a73-8982-933159c46770 service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:31.302376 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdebbb0-71af-4a5b-8c55-bcbba4794466 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:31.303480 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdebbb0-71af-4a5b-8c55-bcbba4794466 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:31.303822 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdebbb0-71af-4a5b-8c55-bcbba4794466 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:31.304156 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdebbb0-71af-4a5b-8c55-bcbba4794466 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:31.313963 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdebbb0-71af-4a5b-8c55-bcbba4794466 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:31.315272 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdebbb0-71af-4a5b-8c55-bcbba4794466 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:31.325082 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1260/2535] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:25:31 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:25:38.936271 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c57d3e-9f1f-4613-8d0d-b4fac5559663 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:38.936939 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c57d3e-9f1f-4613-8d0d-b4fac5559663 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:38.937066 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c57d3e-9f1f-4613-8d0d-b4fac5559663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:38.970967 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-f6c57d3e-9f1f-4613-8d0d-b4fac5559663 None None] MFA Rules not processed for user `d865006413b24ec3922619311f81f353`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:25:38.972405 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f6c57d3e-9f1f-4613-8d0d-b4fac5559663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:39.023696 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:25:39.045039 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1276/2536] 10.222.0.43 () {58 vars in 1009 bytes} [Tue Sep 6 16:25:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:25:39.108280 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdebbb0-71af-4a5b-8c55-bcbba4794466 service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:39.109926 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-4bdebbb0-71af-4a5b-8c55-bcbba4794466 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:39.255906 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdebbb0-71af-4a5b-8c55-bcbba4794466 service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:39.261608 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d376c0b-ea14-4f58-b01a-0ed9e6615e34 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '312026c080b64aff9dd483d3591288a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:39.262309 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d376c0b-ea14-4f58-b01a-0ed9e6615e34 service swift] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:39.262652 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d376c0b-ea14-4f58-b01a-0ed9e6615e34 service swift] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:39.262983 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d376c0b-ea14-4f58-b01a-0ed9e6615e34 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:39.274067 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d376c0b-ea14-4f58-b01a-0ed9e6615e34 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:39.276108 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d376c0b-ea14-4f58-b01a-0ed9e6615e34 service swift] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:39.287789 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1261/2537] 10.222.0.43 () {60 vars in 1384 bytes} [Tue Sep 6 16:25:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 16:25:45.763494 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c57d3e-9f1f-4613-8d0d-b4fac5559663 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:45.848975 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c57d3e-9f1f-4613-8d0d-b4fac5559663 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:45.854460 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee101ede-09b0-4feb-885d-28875c3d836f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:45.855216 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee101ede-09b0-4feb-885d-28875c3d836f service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:45.855590 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee101ede-09b0-4feb-885d-28875c3d836f service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:45.855959 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee101ede-09b0-4feb-885d-28875c3d836f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:45.858221 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ee101ede-09b0-4feb-885d-28875c3d836f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:46.029862 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee101ede-09b0-4feb-885d-28875c3d836f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:46.032033 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee101ede-09b0-4feb-885d-28875c3d836f service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:46.051100 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1277/2538] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:25:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:25:46.710447 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d376c0b-ea14-4f58-b01a-0ed9e6615e34 service swift] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:46.723901 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:25:46.784019 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d376c0b-ea14-4f58-b01a-0ed9e6615e34 service swift] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:46.792435 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774f1f59-f154-4a8a-b38f-037b9e6f6b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:46.793878 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774f1f59-f154-4a8a-b38f-037b9e6f6b07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:46.794568 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774f1f59-f154-4a8a-b38f-037b9e6f6b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:46.795925 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774f1f59-f154-4a8a-b38f-037b9e6f6b07 admin admin] PATH_INFO: `/v3/users/2c0df058d6c946b88186c35d09a17b78` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:46.798881 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774f1f59-f154-4a8a-b38f-037b9e6f6b07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c0df058d6c946b88186c35d09a17b78)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:46.803119 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774f1f59-f154-4a8a-b38f-037b9e6f6b07 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:46.892941 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-774f1f59-f154-4a8a-b38f-037b9e6f6b07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c0df058d6c946b88186c35d09a17b78'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:46.897358 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-774f1f59-f154-4a8a-b38f-037b9e6f6b07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c0df058d6c946b88186c35d09a17b78'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:46.908483 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-774f1f59-f154-4a8a-b38f-037b9e6f6b07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c0df058d6c946b88186c35d09a17b78'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:46.932154 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-774f1f59-f154-4a8a-b38f-037b9e6f6b07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c0df058d6c946b88186c35d09a17b78'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:46.946238 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-774f1f59-f154-4a8a-b38f-037b9e6f6b07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c0df058d6c946b88186c35d09a17b78'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:46.955163 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1262/2539] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:25:46 2022] DELETE /identity/v3/users/2c0df058d6c946b88186c35d09a17b78 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:25:47.386696 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee101ede-09b0-4feb-885d-28875c3d836f service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:47.401023 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee101ede-09b0-4feb-885d-28875c3d836f service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:47.543878 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c039895d-6684-4ad4-9af2-6854d2e636ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:47.544622 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c039895d-6684-4ad4-9af2-6854d2e636ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:47.544743 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c039895d-6684-4ad4-9af2-6854d2e636ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:47.545123 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c039895d-6684-4ad4-9af2-6854d2e636ab admin admin] PATH_INFO: `/v3/projects/ddade75696454ef7bd7794c4e70370fb` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:47.546558 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c039895d-6684-4ad4-9af2-6854d2e636ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddade75696454ef7bd7794c4e70370fb)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:47.548694 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c039895d-6684-4ad4-9af2-6854d2e636ab admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:47.622701 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:25:47.643115 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-c039895d-6684-4ad4-9af2-6854d2e636ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddade75696454ef7bd7794c4e70370fb', 'request_id': 'req-c039895d-6684-4ad4-9af2-6854d2e636ab'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:47.650672 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-c039895d-6684-4ad4-9af2-6854d2e636ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddade75696454ef7bd7794c4e70370fb', 'request_id': 'req-c039895d-6684-4ad4-9af2-6854d2e636ab'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:47.665177 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-c039895d-6684-4ad4-9af2-6854d2e636ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddade75696454ef7bd7794c4e70370fb', 'request_id': 'req-c039895d-6684-4ad4-9af2-6854d2e636ab'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:47.685283 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-c039895d-6684-4ad4-9af2-6854d2e636ab admin admin] The token cache is being invalidate because project ddade75696454ef7bd7794c4e70370fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:25:47.685775 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-c039895d-6684-4ad4-9af2-6854d2e636ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:47.686743 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-c039895d-6684-4ad4-9af2-6854d2e636ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:47.691945 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1278/2540] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:25:47 2022] DELETE /identity/v3/projects/ddade75696454ef7bd7794c4e70370fb => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:25:47.693281 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774f1f59-f154-4a8a-b38f-037b9e6f6b07 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:47.697648 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-774f1f59-f154-4a8a-b38f-037b9e6f6b07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:47.714794 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774f1f59-f154-4a8a-b38f-037b9e6f6b07 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:47.782441 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4679bd-1eb6-4972-9333-0ed92c95906c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:47.783164 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4679bd-1eb6-4972-9333-0ed92c95906c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:47.783522 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4679bd-1eb6-4972-9333-0ed92c95906c admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:47.783854 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4679bd-1eb6-4972-9333-0ed92c95906c admin admin] PATH_INFO: `/v3/users/978a56bc4be040a997f17214161e15f5` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:47.785173 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4679bd-1eb6-4972-9333-0ed92c95906c admin admin] RBAC: Authorizing `identity:delete_user(user_id=978a56bc4be040a997f17214161e15f5)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:47.787251 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4679bd-1eb6-4972-9333-0ed92c95906c admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:47.844941 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:25:47.860607 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-5d4679bd-1eb6-4972-9333-0ed92c95906c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '978a56bc4be040a997f17214161e15f5'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:47.861518 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-5d4679bd-1eb6-4972-9333-0ed92c95906c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '978a56bc4be040a997f17214161e15f5'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:47.876169 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-5d4679bd-1eb6-4972-9333-0ed92c95906c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '978a56bc4be040a997f17214161e15f5'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:47.896799 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-5d4679bd-1eb6-4972-9333-0ed92c95906c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '978a56bc4be040a997f17214161e15f5'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:47.906419 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-5d4679bd-1eb6-4972-9333-0ed92c95906c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '978a56bc4be040a997f17214161e15f5'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:47.913700 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1263/2541] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:25:47 2022] DELETE /identity/v3/users/978a56bc4be040a997f17214161e15f5 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:25:48.139093 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c039895d-6684-4ad4-9af2-6854d2e636ab admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:48.141539 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-c039895d-6684-4ad4-9af2-6854d2e636ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:48.159312 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c039895d-6684-4ad4-9af2-6854d2e636ab admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:48.283445 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92d22b9-151e-48fd-a15b-21f81bab3449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:48.284220 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92d22b9-151e-48fd-a15b-21f81bab3449 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:48.284596 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92d22b9-151e-48fd-a15b-21f81bab3449 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:48.284960 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92d22b9-151e-48fd-a15b-21f81bab3449 admin admin] PATH_INFO: `/v3/projects/46f1d0af2b1d434886dfb02672df9681` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:48.286335 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92d22b9-151e-48fd-a15b-21f81bab3449 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46f1d0af2b1d434886dfb02672df9681)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:48.289085 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92d22b9-151e-48fd-a15b-21f81bab3449 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:48.366210 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d92d22b9-151e-48fd-a15b-21f81bab3449 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46f1d0af2b1d434886dfb02672df9681', 'request_id': 'req-d92d22b9-151e-48fd-a15b-21f81bab3449'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:48.373801 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d92d22b9-151e-48fd-a15b-21f81bab3449 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46f1d0af2b1d434886dfb02672df9681', 'request_id': 'req-d92d22b9-151e-48fd-a15b-21f81bab3449'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:48.391346 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d92d22b9-151e-48fd-a15b-21f81bab3449 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46f1d0af2b1d434886dfb02672df9681', 'request_id': 'req-d92d22b9-151e-48fd-a15b-21f81bab3449'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:48.408469 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d92d22b9-151e-48fd-a15b-21f81bab3449 admin admin] The token cache is being invalidate because project 46f1d0af2b1d434886dfb02672df9681 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:25:48.408747 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d92d22b9-151e-48fd-a15b-21f81bab3449 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:48.409894 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d92d22b9-151e-48fd-a15b-21f81bab3449 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:48.412674 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1279/2542] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:25:48 2022] DELETE /identity/v3/projects/46f1d0af2b1d434886dfb02672df9681 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:25:48.417912 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f719b3-0856-4895-80f6-c76fca90fd4b None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:48.418526 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f719b3-0856-4895-80f6-c76fca90fd4b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:48.419011 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f719b3-0856-4895-80f6-c76fca90fd4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:48.459553 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92d22b9-151e-48fd-a15b-21f81bab3449 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:48.461163 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d92d22b9-151e-48fd-a15b-21f81bab3449 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:48.463397 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-e7f719b3-0856-4895-80f6-c76fca90fd4b None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:25:48.464874 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e7f719b3-0856-4895-80f6-c76fca90fd4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:48.537784 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:25:48.547944 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92d22b9-151e-48fd-a15b-21f81bab3449 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:48.554864 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a7205f-53bd-4505-ac38-5b54cf89062b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:48.555763 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a7205f-53bd-4505-ac38-5b54cf89062b service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:48.556198 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a7205f-53bd-4505-ac38-5b54cf89062b service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:48.556621 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a7205f-53bd-4505-ac38-5b54cf89062b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:48.558699 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-73a7205f-53bd-4505-ac38-5b54cf89062b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:48.560449 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1264/2543] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:25:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:25:48.562190 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca7e20c-8984-4f07-82ec-03c95b8eebe4 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:48.562952 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca7e20c-8984-4f07-82ec-03c95b8eebe4 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:48.564039 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca7e20c-8984-4f07-82ec-03c95b8eebe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:48.622614 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-6ca7e20c-8984-4f07-82ec-03c95b8eebe4 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:25:48.627480 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-6ca7e20c-8984-4f07-82ec-03c95b8eebe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:48.660845 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a7205f-53bd-4505-ac38-5b54cf89062b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:48.661770 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1265/2544] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:25:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:25:48.661997 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a7205f-53bd-4505-ac38-5b54cf89062b service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:48.663861 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca7e20c-8984-4f07-82ec-03c95b8eebe4 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:48.686681 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca7e20c-8984-4f07-82ec-03c95b8eebe4 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:48.686977 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1280/2545] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:25:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:25:48.707130 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c247deab-887f-49a7-a9c7-69583ab5eff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:48.707772 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c247deab-887f-49a7-a9c7-69583ab5eff6 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:48.708028 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c247deab-887f-49a7-a9c7-69583ab5eff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:48.708293 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c247deab-887f-49a7-a9c7-69583ab5eff6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:48.708975 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c247deab-887f-49a7-a9c7-69583ab5eff6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:25:48.710044 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c247deab-887f-49a7-a9c7-69583ab5eff6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:48.710898 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c247deab-887f-49a7-a9c7-69583ab5eff6 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:48.740552 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1266/2546] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 6 16:25:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:25:48.742084 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a7205f-53bd-4505-ac38-5b54cf89062b service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:48.764572 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a7205f-53bd-4505-ac38-5b54cf89062b service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:48.770248 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febef0cc-7ed3-45d0-9c0d-f243ea137010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:48.771160 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febef0cc-7ed3-45d0-9c0d-f243ea137010 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:48.771588 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febef0cc-7ed3-45d0-9c0d-f243ea137010 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:48.772001 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febef0cc-7ed3-45d0-9c0d-f243ea137010 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:48.772840 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febef0cc-7ed3-45d0-9c0d-f243ea137010 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:48.775109 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febef0cc-7ed3-45d0-9c0d-f243ea137010 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:48.829657 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1281/2547] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:25:48 2022] POST /identity/v3/projects => generated 373 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:25:48.835067 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c247deab-887f-49a7-a9c7-69583ab5eff6 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:48.849355 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c247deab-887f-49a7-a9c7-69583ab5eff6 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:48.969925 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83da658d-564e-4212-bdc2-73a4aff0cf3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:48.970609 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83da658d-564e-4212-bdc2-73a4aff0cf3e admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:48.970887 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83da658d-564e-4212-bdc2-73a4aff0cf3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:48.971170 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83da658d-564e-4212-bdc2-73a4aff0cf3e admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:48.971858 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83da658d-564e-4212-bdc2-73a4aff0cf3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:48.973922 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83da658d-564e-4212-bdc2-73a4aff0cf3e admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:49.005365 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1267/2548] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:25:48 2022] POST /identity/v3/users => generated 287 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:25:49.010910 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febef0cc-7ed3-45d0-9c0d-f243ea137010 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:49.027957 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febef0cc-7ed3-45d0-9c0d-f243ea137010 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:49.033424 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84709c8b-6d18-4aa0-a910-cb740456a3fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:49.034275 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84709c8b-6d18-4aa0-a910-cb740456a3fc admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:49.034564 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84709c8b-6d18-4aa0-a910-cb740456a3fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:49.034832 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84709c8b-6d18-4aa0-a910-cb740456a3fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:49.035657 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84709c8b-6d18-4aa0-a910-cb740456a3fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:49.037331 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84709c8b-6d18-4aa0-a910-cb740456a3fc admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:49.054986 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1282/2549] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:25:49 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:25:49.060392 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83da658d-564e-4212-bdc2-73a4aff0cf3e admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:49.072197 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83da658d-564e-4212-bdc2-73a4aff0cf3e admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:49.078267 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c917017-1f18-47d9-ab33-2291f31eff95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:49.079883 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c917017-1f18-47d9-ab33-2291f31eff95 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:49.080199 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c917017-1f18-47d9-ab33-2291f31eff95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:49.081042 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c917017-1f18-47d9-ab33-2291f31eff95 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:49.083181 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c917017-1f18-47d9-ab33-2291f31eff95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:49.086092 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c917017-1f18-47d9-ab33-2291f31eff95 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:49.100648 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1268/2550] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:25:49 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:25:49.101956 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84709c8b-6d18-4aa0-a910-cb740456a3fc admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:49.113641 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84709c8b-6d18-4aa0-a910-cb740456a3fc admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:49.119004 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68bc00d-1f1e-40e3-b052-91d07d5d533e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:49.119798 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68bc00d-1f1e-40e3-b052-91d07d5d533e admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:49.120398 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68bc00d-1f1e-40e3-b052-91d07d5d533e admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:49.120781 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68bc00d-1f1e-40e3-b052-91d07d5d533e admin admin] PATH_INFO: `/v3/projects/cc384f7a9bf44d29b641308f1358817a/users/6ca8a60e15744f6396cce0ff3dacf72b/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:49.140090 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68bc00d-1f1e-40e3-b052-91d07d5d533e admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc384f7a9bf44d29b641308f1358817a, user_id=6ca8a60e15744f6396cce0ff3dacf72b, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:49.143411 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68bc00d-1f1e-40e3-b052-91d07d5d533e admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:49.163262 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1283/2551] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:25:49 2022] PUT /identity/v3/projects/cc384f7a9bf44d29b641308f1358817a/users/6ca8a60e15744f6396cce0ff3dacf72b/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:25:49.176166 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6bcc71-be93-4dde-b071-51d171a14c96 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:49.176920 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6bcc71-be93-4dde-b071-51d171a14c96 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:49.177746 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6bcc71-be93-4dde-b071-51d171a14c96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:49.204945 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:25:49.230415 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-cf6bcc71-be93-4dde-b071-51d171a14c96 None None] MFA Rules not processed for user `6ca8a60e15744f6396cce0ff3dacf72b`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:25:49.236363 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-cf6bcc71-be93-4dde-b071-51d171a14c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:49.359523 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1269/2552] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:25:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:25:49.382444 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68bc00d-1f1e-40e3-b052-91d07d5d533e admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:49.385748 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e68bc00d-1f1e-40e3-b052-91d07d5d533e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:49.473236 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68bc00d-1f1e-40e3-b052-91d07d5d533e admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:49.477792 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c382da0-9e2d-423c-90c1-7b8c881a49a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:49.478437 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c382da0-9e2d-423c-90c1-7b8c881a49a1 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:49.478711 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c382da0-9e2d-423c-90c1-7b8c881a49a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:49.478976 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c382da0-9e2d-423c-90c1-7b8c881a49a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:49.491711 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c382da0-9e2d-423c-90c1-7b8c881a49a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:49.493265 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c382da0-9e2d-423c-90c1-7b8c881a49a1 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:49.503318 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1284/2553] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:25:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:25:49.525590 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6bcc71-be93-4dde-b071-51d171a14c96 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:49.635477 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6bcc71-be93-4dde-b071-51d171a14c96 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:49.641848 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77576fa-fbe6-436d-96ea-4c6d62bd13b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:49.642535 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77576fa-fbe6-436d-96ea-4c6d62bd13b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:49.642804 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77576fa-fbe6-436d-96ea-4c6d62bd13b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:49.643069 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77576fa-fbe6-436d-96ea-4c6d62bd13b8 admin admin] PATH_INFO: `/v3/users/6ca8a60e15744f6396cce0ff3dacf72b` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:49.644216 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77576fa-fbe6-436d-96ea-4c6d62bd13b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ca8a60e15744f6396cce0ff3dacf72b)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:49.646156 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77576fa-fbe6-436d-96ea-4c6d62bd13b8 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:49.719296 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d77576fa-fbe6-436d-96ea-4c6d62bd13b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ca8a60e15744f6396cce0ff3dacf72b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:49.720264 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d77576fa-fbe6-436d-96ea-4c6d62bd13b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ca8a60e15744f6396cce0ff3dacf72b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:49.731292 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d77576fa-fbe6-436d-96ea-4c6d62bd13b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ca8a60e15744f6396cce0ff3dacf72b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:49.753910 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d77576fa-fbe6-436d-96ea-4c6d62bd13b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ca8a60e15744f6396cce0ff3dacf72b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:49.763472 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d77576fa-fbe6-436d-96ea-4c6d62bd13b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ca8a60e15744f6396cce0ff3dacf72b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:49.772326 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1270/2554] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:25:49 2022] DELETE /identity/v3/users/6ca8a60e15744f6396cce0ff3dacf72b => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:25:49.778189 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f5b096-e441-43ce-b630-c54332955a0c None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:49.778596 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f5b096-e441-43ce-b630-c54332955a0c None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:49.778919 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f5b096-e441-43ce-b630-c54332955a0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:49.808193 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-c8f5b096-e441-43ce-b630-c54332955a0c None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:25:49.809549 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-c8f5b096-e441-43ce-b630-c54332955a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:49.841928 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:25:49.894530 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1285/2555] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:25:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:25:49.905878 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77576fa-fbe6-436d-96ea-4c6d62bd13b8 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:49.907530 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d77576fa-fbe6-436d-96ea-4c6d62bd13b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:49.940961 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:25:50.010049 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77576fa-fbe6-436d-96ea-4c6d62bd13b8 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:50.016694 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d40d7ca-f425-4296-88de-a46e038ee4a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:50.018975 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d40d7ca-f425-4296-88de-a46e038ee4a8 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:50.019255 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d40d7ca-f425-4296-88de-a46e038ee4a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:50.019526 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d40d7ca-f425-4296-88de-a46e038ee4a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:50.033101 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d40d7ca-f425-4296-88de-a46e038ee4a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:50.034753 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d40d7ca-f425-4296-88de-a46e038ee4a8 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:50.051190 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1271/2556] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:25:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:25:50.440009 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f5b096-e441-43ce-b630-c54332955a0c None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:50.441662 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-c8f5b096-e441-43ce-b630-c54332955a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:50.455453 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f5b096-e441-43ce-b630-c54332955a0c None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:50.460090 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c66e772-bf59-472e-9760-fafc605ddd40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:50.460732 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c66e772-bf59-472e-9760-fafc605ddd40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:50.461051 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c66e772-bf59-472e-9760-fafc605ddd40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:50.461318 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c66e772-bf59-472e-9760-fafc605ddd40 admin admin] PATH_INFO: `/v3/projects/cc384f7a9bf44d29b641308f1358817a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:50.462427 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c66e772-bf59-472e-9760-fafc605ddd40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc384f7a9bf44d29b641308f1358817a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:50.464403 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c66e772-bf59-472e-9760-fafc605ddd40 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:50.514686 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2c66e772-bf59-472e-9760-fafc605ddd40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc384f7a9bf44d29b641308f1358817a', 'request_id': 'req-2c66e772-bf59-472e-9760-fafc605ddd40'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:50.521699 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2c66e772-bf59-472e-9760-fafc605ddd40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc384f7a9bf44d29b641308f1358817a', 'request_id': 'req-2c66e772-bf59-472e-9760-fafc605ddd40'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:50.531308 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2c66e772-bf59-472e-9760-fafc605ddd40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc384f7a9bf44d29b641308f1358817a', 'request_id': 'req-2c66e772-bf59-472e-9760-fafc605ddd40'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:50.542849 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2c66e772-bf59-472e-9760-fafc605ddd40 admin admin] The token cache is being invalidate because project cc384f7a9bf44d29b641308f1358817a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:25:50.543116 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2c66e772-bf59-472e-9760-fafc605ddd40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:50.543814 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2c66e772-bf59-472e-9760-fafc605ddd40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:50.545357 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1286/2557] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:25:50 2022] DELETE /identity/v3/projects/cc384f7a9bf44d29b641308f1358817a => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:25:50.550550 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50904dd-e6dc-41b9-be83-09d6ed90ad02 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:50.550893 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50904dd-e6dc-41b9-be83-09d6ed90ad02 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:50.551223 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50904dd-e6dc-41b9-be83-09d6ed90ad02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:50.574292 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-e50904dd-e6dc-41b9-be83-09d6ed90ad02 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:25:50.575676 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e50904dd-e6dc-41b9-be83-09d6ed90ad02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:50.639441 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1272/2558] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:25:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:25:50.708632 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aa8a11-b1c3-4c5e-9446-94d96e76c809 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:50.709089 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aa8a11-b1c3-4c5e-9446-94d96e76c809 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:50.709574 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aa8a11-b1c3-4c5e-9446-94d96e76c809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:50.731169 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-a5aa8a11-b1c3-4c5e-9446-94d96e76c809 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:25:50.732312 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a5aa8a11-b1c3-4c5e-9446-94d96e76c809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:50.741442 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1287/2559] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:25:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:25:50.746136 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50904dd-e6dc-41b9-be83-09d6ed90ad02 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:50.758866 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50904dd-e6dc-41b9-be83-09d6ed90ad02 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:50.767837 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b606d7-2576-42cf-b1ff-3d5524c4762c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:50.768385 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b606d7-2576-42cf-b1ff-3d5524c4762c admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:50.768665 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b606d7-2576-42cf-b1ff-3d5524c4762c admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:50.768963 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b606d7-2576-42cf-b1ff-3d5524c4762c admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:50.769611 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b606d7-2576-42cf-b1ff-3d5524c4762c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:25:50.769913 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b606d7-2576-42cf-b1ff-3d5524c4762c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:50.771658 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b606d7-2576-42cf-b1ff-3d5524c4762c admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:50.784414 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1273/2560] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 6 16:25:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:25:50.790548 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aa8a11-b1c3-4c5e-9446-94d96e76c809 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:50.801071 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aa8a11-b1c3-4c5e-9446-94d96e76c809 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:50.807148 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009e3047-4982-4d58-a98f-650f75d60251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:50.807751 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009e3047-4982-4d58-a98f-650f75d60251 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:50.808031 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009e3047-4982-4d58-a98f-650f75d60251 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:50.808302 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009e3047-4982-4d58-a98f-650f75d60251 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:50.808965 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009e3047-4982-4d58-a98f-650f75d60251 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:50.810899 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009e3047-4982-4d58-a98f-650f75d60251 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:50.841343 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1288/2561] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:25:50 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:25:50.846080 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b606d7-2576-42cf-b1ff-3d5524c4762c admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:50.857422 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b606d7-2576-42cf-b1ff-3d5524c4762c admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:50.917182 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f2b229-47e8-4ef3-b70b-742fb2795e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:50.917846 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f2b229-47e8-4ef3-b70b-742fb2795e70 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:50.918127 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f2b229-47e8-4ef3-b70b-742fb2795e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:50.918394 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f2b229-47e8-4ef3-b70b-742fb2795e70 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:50.919063 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f2b229-47e8-4ef3-b70b-742fb2795e70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:50.921491 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f2b229-47e8-4ef3-b70b-742fb2795e70 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:50.946994 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1274/2562] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:25:50 2022] POST /identity/v3/users => generated 286 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:25:50.951780 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009e3047-4982-4d58-a98f-650f75d60251 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:50.962748 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009e3047-4982-4d58-a98f-650f75d60251 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:50.970576 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74ff3ec-f4d6-44a8-8c8a-deefd133be9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:50.971288 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74ff3ec-f4d6-44a8-8c8a-deefd133be9e admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:50.971637 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74ff3ec-f4d6-44a8-8c8a-deefd133be9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:50.971943 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74ff3ec-f4d6-44a8-8c8a-deefd133be9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:50.973089 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74ff3ec-f4d6-44a8-8c8a-deefd133be9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:50.974764 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74ff3ec-f4d6-44a8-8c8a-deefd133be9e admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:50.989297 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1289/2563] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:25:50 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:25:50.993687 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f2b229-47e8-4ef3-b70b-742fb2795e70 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:51.007955 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f2b229-47e8-4ef3-b70b-742fb2795e70 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:51.013251 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772544f7-e132-4d9c-85f8-0e0d7a19dffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:51.013890 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772544f7-e132-4d9c-85f8-0e0d7a19dffe admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:51.014163 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772544f7-e132-4d9c-85f8-0e0d7a19dffe admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:51.014461 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772544f7-e132-4d9c-85f8-0e0d7a19dffe admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:51.015115 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772544f7-e132-4d9c-85f8-0e0d7a19dffe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:51.016928 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772544f7-e132-4d9c-85f8-0e0d7a19dffe admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:51.027927 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1275/2564] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:25:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 16:25:51.032746 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74ff3ec-f4d6-44a8-8c8a-deefd133be9e admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:51.043423 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74ff3ec-f4d6-44a8-8c8a-deefd133be9e admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:51.049153 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da466a31-b22a-4358-95dc-84b107758004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:51.049811 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da466a31-b22a-4358-95dc-84b107758004 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:51.050089 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da466a31-b22a-4358-95dc-84b107758004 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:51.050357 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da466a31-b22a-4358-95dc-84b107758004 admin admin] PATH_INFO: `/v3/projects/c854cfc1706249dcaea86f9fe9a235e3/users/3ee682c3cd2845dba42c4ea2bee8825c/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:51.066914 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da466a31-b22a-4358-95dc-84b107758004 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c854cfc1706249dcaea86f9fe9a235e3, user_id=3ee682c3cd2845dba42c4ea2bee8825c, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:51.069626 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da466a31-b22a-4358-95dc-84b107758004 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:51.083033 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1290/2565] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:25:51 2022] PUT /identity/v3/projects/c854cfc1706249dcaea86f9fe9a235e3/users/3ee682c3cd2845dba42c4ea2bee8825c/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:25:51.094230 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018835ca-6841-4a59-80f0-82bd8e9cf6e0 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:51.094545 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018835ca-6841-4a59-80f0-82bd8e9cf6e0 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:51.094834 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018835ca-6841-4a59-80f0-82bd8e9cf6e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:51.125502 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-018835ca-6841-4a59-80f0-82bd8e9cf6e0 None None] MFA Rules not processed for user `3ee682c3cd2845dba42c4ea2bee8825c`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:25:51.126762 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-018835ca-6841-4a59-80f0-82bd8e9cf6e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:51.144996 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:25:51.200608 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1276/2566] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:25:51 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:25:51.201962 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da466a31-b22a-4358-95dc-84b107758004 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:51.269929 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:25:51.300025 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da466a31-b22a-4358-95dc-84b107758004 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:51.306650 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26778d41-d6eb-4543-817f-07eaaa9ae12d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:51.307446 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26778d41-d6eb-4543-817f-07eaaa9ae12d admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:51.307829 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26778d41-d6eb-4543-817f-07eaaa9ae12d admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:51.308519 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26778d41-d6eb-4543-817f-07eaaa9ae12d admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:51.309297 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26778d41-d6eb-4543-817f-07eaaa9ae12d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:51.311462 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26778d41-d6eb-4543-817f-07eaaa9ae12d admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:51.368552 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1291/2567] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:25:51 2022] POST /identity/v3/projects => generated 373 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:25:51.373657 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018835ca-6841-4a59-80f0-82bd8e9cf6e0 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:51.394103 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018835ca-6841-4a59-80f0-82bd8e9cf6e0 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:51.466105 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b100e8f4-8220-4cec-a7f0-19369a33012a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:51.466782 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b100e8f4-8220-4cec-a7f0-19369a33012a admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:51.467059 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b100e8f4-8220-4cec-a7f0-19369a33012a admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:51.467329 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b100e8f4-8220-4cec-a7f0-19369a33012a admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:51.467996 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b100e8f4-8220-4cec-a7f0-19369a33012a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:51.469977 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b100e8f4-8220-4cec-a7f0-19369a33012a admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:51.491895 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1277/2568] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:25:51 2022] POST /identity/v3/users => generated 293 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:25:51.496065 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26778d41-d6eb-4543-817f-07eaaa9ae12d admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:51.506542 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26778d41-d6eb-4543-817f-07eaaa9ae12d admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:51.511424 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aa7c1c-e72b-45cd-a7a9-aad7ec773e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:51.512015 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa7c1c-e72b-45cd-a7a9-aad7ec773e1a admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:51.512267 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa7c1c-e72b-45cd-a7a9-aad7ec773e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:51.512563 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa7c1c-e72b-45cd-a7a9-aad7ec773e1a admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:51.513231 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aa7c1c-e72b-45cd-a7a9-aad7ec773e1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:51.514917 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aa7c1c-e72b-45cd-a7a9-aad7ec773e1a admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:51.527985 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1292/2569] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:25:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 16:25:51.532688 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b100e8f4-8220-4cec-a7f0-19369a33012a admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:51.543336 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b100e8f4-8220-4cec-a7f0-19369a33012a admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:51.549213 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7785dc5f-12ef-482d-a0bd-73280d7f50df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:51.549865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7785dc5f-12ef-482d-a0bd-73280d7f50df admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:51.550138 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7785dc5f-12ef-482d-a0bd-73280d7f50df admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:51.550398 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7785dc5f-12ef-482d-a0bd-73280d7f50df admin admin] PATH_INFO: `/v3/projects/a203340651a44f828f87953fae9bcbc4/users/12d859dffcf842f58feb2bd81f727bbb/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:51.566577 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7785dc5f-12ef-482d-a0bd-73280d7f50df admin admin] RBAC: Authorizing `identity:create_grant(project_id=a203340651a44f828f87953fae9bcbc4, user_id=12d859dffcf842f58feb2bd81f727bbb, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:51.569250 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7785dc5f-12ef-482d-a0bd-73280d7f50df admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:51.582660 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1278/2570] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:25:51 2022] PUT /identity/v3/projects/a203340651a44f828f87953fae9bcbc4/users/12d859dffcf842f58feb2bd81f727bbb/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:25:51.587316 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aa7c1c-e72b-45cd-a7a9-aad7ec773e1a admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:51.602911 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aa7c1c-e72b-45cd-a7a9-aad7ec773e1a admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:51.661762 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafc104b-891c-4a4e-ade6-83100faf66ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:51.662296 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc104b-891c-4a4e-ade6-83100faf66ee admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:51.662577 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc104b-891c-4a4e-ade6-83100faf66ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:51.662846 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc104b-891c-4a4e-ade6-83100faf66ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:51.663531 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafc104b-891c-4a4e-ade6-83100faf66ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:51.665462 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafc104b-891c-4a4e-ade6-83100faf66ee admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:51.676067 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1293/2571] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:25:51 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:25:51.680883 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7785dc5f-12ef-482d-a0bd-73280d7f50df admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:51.692575 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7785dc5f-12ef-482d-a0bd-73280d7f50df admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:51.697314 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1392b5be-7766-48de-8f76-6bed712957b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:51.698003 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1392b5be-7766-48de-8f76-6bed712957b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:51.698274 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1392b5be-7766-48de-8f76-6bed712957b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:51.698544 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1392b5be-7766-48de-8f76-6bed712957b0 admin admin] PATH_INFO: `/v3/projects/a203340651a44f828f87953fae9bcbc4/users/12d859dffcf842f58feb2bd81f727bbb/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:51.700554 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1392b5be-7766-48de-8f76-6bed712957b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a203340651a44f828f87953fae9bcbc4, user_id=12d859dffcf842f58feb2bd81f727bbb, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:51.703228 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1392b5be-7766-48de-8f76-6bed712957b0 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:51.714827 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1279/2572] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:25:51 2022] PUT /identity/v3/projects/a203340651a44f828f87953fae9bcbc4/users/12d859dffcf842f58feb2bd81f727bbb/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:25:51.725613 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c3e73b-91b4-4ce6-a54d-f4063eae8136 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:51.725896 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c3e73b-91b4-4ce6-a54d-f4063eae8136 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:51.726166 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c3e73b-91b4-4ce6-a54d-f4063eae8136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:51.755226 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-d8c3e73b-91b4-4ce6-a54d-f4063eae8136 None None] MFA Rules not processed for user `12d859dffcf842f58feb2bd81f727bbb`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:25:51.756492 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d8c3e73b-91b4-4ce6-a54d-f4063eae8136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:51.825375 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1294/2573] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:25:51 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:25:51.836455 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1392b5be-7766-48de-8f76-6bed712957b0 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:51.838295 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-1392b5be-7766-48de-8f76-6bed712957b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:51.898588 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1392b5be-7766-48de-8f76-6bed712957b0 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:51.902647 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ba68bc-62e4-4c1b-9e3d-0975d6d5f20e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:51.903263 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ba68bc-62e4-4c1b-9e3d-0975d6d5f20e service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:51.903537 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ba68bc-62e4-4c1b-9e3d-0975d6d5f20e service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:51.903791 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ba68bc-62e4-4c1b-9e3d-0975d6d5f20e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:51.929528 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:25:51.962266 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ba68bc-62e4-4c1b-9e3d-0975d6d5f20e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:51.963707 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ba68bc-62e4-4c1b-9e3d-0975d6d5f20e service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:51.973253 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1280/2574] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:25:51 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:25:54.587274 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c3e73b-91b4-4ce6-a54d-f4063eae8136 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:54.644368 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:25:54.673746 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c3e73b-91b4-4ce6-a54d-f4063eae8136 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:54.683300 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427c0b4b-902a-410d-bd06-991a9ce850eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:54.684023 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427c0b4b-902a-410d-bd06-991a9ce850eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:54.684272 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427c0b4b-902a-410d-bd06-991a9ce850eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:54.684529 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427c0b4b-902a-410d-bd06-991a9ce850eb admin admin] PATH_INFO: `/v3/users/3ee682c3cd2845dba42c4ea2bee8825c` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:54.685803 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427c0b4b-902a-410d-bd06-991a9ce850eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ee682c3cd2845dba42c4ea2bee8825c)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:54.688922 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427c0b4b-902a-410d-bd06-991a9ce850eb admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:54.761017 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-427c0b4b-902a-410d-bd06-991a9ce850eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ee682c3cd2845dba42c4ea2bee8825c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:54.762563 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-427c0b4b-902a-410d-bd06-991a9ce850eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ee682c3cd2845dba42c4ea2bee8825c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:54.773457 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-427c0b4b-902a-410d-bd06-991a9ce850eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ee682c3cd2845dba42c4ea2bee8825c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:54.787759 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-427c0b4b-902a-410d-bd06-991a9ce850eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ee682c3cd2845dba42c4ea2bee8825c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:54.788801 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-427c0b4b-902a-410d-bd06-991a9ce850eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ee682c3cd2845dba42c4ea2bee8825c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:54.804628 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1295/2575] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:25:54 2022] DELETE /identity/v3/users/3ee682c3cd2845dba42c4ea2bee8825c => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:25:54.809850 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df67310-b45d-4257-bab1-2b5e7beea25f None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:54.810389 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df67310-b45d-4257-bab1-2b5e7beea25f None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:54.810389 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df67310-b45d-4257-bab1-2b5e7beea25f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:54.834235 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-7df67310-b45d-4257-bab1-2b5e7beea25f None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:25:54.835586 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7df67310-b45d-4257-bab1-2b5e7beea25f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:54.947166 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1281/2576] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:25:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:25:54.960626 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427c0b4b-902a-410d-bd06-991a9ce850eb admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:54.962480 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-427c0b4b-902a-410d-bd06-991a9ce850eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:55.043998 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427c0b4b-902a-410d-bd06-991a9ce850eb admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:55.048198 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4dec5b-101f-40b6-81e6-84736c13a752 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:55.049235 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4dec5b-101f-40b6-81e6-84736c13a752 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:55.049663 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4dec5b-101f-40b6-81e6-84736c13a752 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:55.050108 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4dec5b-101f-40b6-81e6-84736c13a752 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:55.063843 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4dec5b-101f-40b6-81e6-84736c13a752 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:55.065646 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4dec5b-101f-40b6-81e6-84736c13a752 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:55.081974 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1296/2577] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:25:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:25:55.556060 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df67310-b45d-4257-bab1-2b5e7beea25f None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:55.557569 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7df67310-b45d-4257-bab1-2b5e7beea25f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:55.577722 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df67310-b45d-4257-bab1-2b5e7beea25f None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:55.591325 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d0f817-e695-42dc-a4ab-91c3c00128bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:55.594167 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d0f817-e695-42dc-a4ab-91c3c00128bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:55.594472 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d0f817-e695-42dc-a4ab-91c3c00128bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:55.594816 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d0f817-e695-42dc-a4ab-91c3c00128bb admin admin] PATH_INFO: `/v3/projects/c854cfc1706249dcaea86f9fe9a235e3` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:55.597526 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d0f817-e695-42dc-a4ab-91c3c00128bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=c854cfc1706249dcaea86f9fe9a235e3)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:55.601632 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d0f817-e695-42dc-a4ab-91c3c00128bb admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:55.672303 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-32d0f817-e695-42dc-a4ab-91c3c00128bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c854cfc1706249dcaea86f9fe9a235e3', 'request_id': 'req-32d0f817-e695-42dc-a4ab-91c3c00128bb'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:55.685015 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-32d0f817-e695-42dc-a4ab-91c3c00128bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c854cfc1706249dcaea86f9fe9a235e3', 'request_id': 'req-32d0f817-e695-42dc-a4ab-91c3c00128bb'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:55.693651 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-32d0f817-e695-42dc-a4ab-91c3c00128bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c854cfc1706249dcaea86f9fe9a235e3', 'request_id': 'req-32d0f817-e695-42dc-a4ab-91c3c00128bb'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:55.709117 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-32d0f817-e695-42dc-a4ab-91c3c00128bb admin admin] The token cache is being invalidate because project c854cfc1706249dcaea86f9fe9a235e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:25:55.709411 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-32d0f817-e695-42dc-a4ab-91c3c00128bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:55.710076 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-32d0f817-e695-42dc-a4ab-91c3c00128bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:55.712644 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1282/2578] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:25:55 2022] DELETE /identity/v3/projects/c854cfc1706249dcaea86f9fe9a235e3 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:25:55.716838 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4dec5b-101f-40b6-81e6-84736c13a752 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:55.731248 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4dec5b-101f-40b6-81e6-84736c13a752 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:55.808950 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b83785f-ebdb-4b82-a4b5-e7eabf6bd4df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:55.810084 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b83785f-ebdb-4b82-a4b5-e7eabf6bd4df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:55.810358 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b83785f-ebdb-4b82-a4b5-e7eabf6bd4df admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:55.810625 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b83785f-ebdb-4b82-a4b5-e7eabf6bd4df admin admin] PATH_INFO: `/v3/users/12d859dffcf842f58feb2bd81f727bbb` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:55.811891 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b83785f-ebdb-4b82-a4b5-e7eabf6bd4df admin admin] RBAC: Authorizing `identity:delete_user(user_id=12d859dffcf842f58feb2bd81f727bbb)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:55.813923 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b83785f-ebdb-4b82-a4b5-e7eabf6bd4df admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:55.820470 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:25:55.885674 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6b83785f-ebdb-4b82-a4b5-e7eabf6bd4df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12d859dffcf842f58feb2bd81f727bbb'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:55.886492 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6b83785f-ebdb-4b82-a4b5-e7eabf6bd4df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12d859dffcf842f58feb2bd81f727bbb'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:55.898427 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6b83785f-ebdb-4b82-a4b5-e7eabf6bd4df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12d859dffcf842f58feb2bd81f727bbb'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:55.925382 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6b83785f-ebdb-4b82-a4b5-e7eabf6bd4df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12d859dffcf842f58feb2bd81f727bbb'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:55.926231 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6b83785f-ebdb-4b82-a4b5-e7eabf6bd4df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12d859dffcf842f58feb2bd81f727bbb'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:55.944444 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1297/2579] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:25:55 2022] DELETE /identity/v3/users/12d859dffcf842f58feb2bd81f727bbb => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:25:56.250426 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d0f817-e695-42dc-a4ab-91c3c00128bb admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:56.252124 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-32d0f817-e695-42dc-a4ab-91c3c00128bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:56.285722 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d0f817-e695-42dc-a4ab-91c3c00128bb admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:56.338187 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:25:56.356419 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b83785f-ebdb-4b82-a4b5-e7eabf6bd4df admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:56.356641 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458344ae-7f5e-47e0-bf2f-12184b35a06e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:56.357402 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458344ae-7f5e-47e0-bf2f-12184b35a06e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:56.357798 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458344ae-7f5e-47e0-bf2f-12184b35a06e admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:56.358364 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6b83785f-ebdb-4b82-a4b5-e7eabf6bd4df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:56.358483 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458344ae-7f5e-47e0-bf2f-12184b35a06e admin admin] PATH_INFO: `/v3/projects/a203340651a44f828f87953fae9bcbc4` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:56.360348 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458344ae-7f5e-47e0-bf2f-12184b35a06e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a203340651a44f828f87953fae9bcbc4)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:56.362374 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458344ae-7f5e-47e0-bf2f-12184b35a06e admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:56.422669 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-458344ae-7f5e-47e0-bf2f-12184b35a06e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a203340651a44f828f87953fae9bcbc4', 'request_id': 'req-458344ae-7f5e-47e0-bf2f-12184b35a06e'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:56.437513 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-458344ae-7f5e-47e0-bf2f-12184b35a06e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a203340651a44f828f87953fae9bcbc4', 'request_id': 'req-458344ae-7f5e-47e0-bf2f-12184b35a06e'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:56.444084 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b83785f-ebdb-4b82-a4b5-e7eabf6bd4df admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:56.449500 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-458344ae-7f5e-47e0-bf2f-12184b35a06e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a203340651a44f828f87953fae9bcbc4', 'request_id': 'req-458344ae-7f5e-47e0-bf2f-12184b35a06e'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:56.461497 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-458344ae-7f5e-47e0-bf2f-12184b35a06e admin admin] The token cache is being invalidate because project a203340651a44f828f87953fae9bcbc4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:25:56.462307 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-458344ae-7f5e-47e0-bf2f-12184b35a06e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:56.462991 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-458344ae-7f5e-47e0-bf2f-12184b35a06e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:25:56.465127 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1283/2580] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:25:56 2022] DELETE /identity/v3/projects/a203340651a44f828f87953fae9bcbc4 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:25:56.465996 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458344ae-7f5e-47e0-bf2f-12184b35a06e admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:56.467393 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-458344ae-7f5e-47e0-bf2f-12184b35a06e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:56.511966 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860af1f4-328c-47ab-919e-d63a8fc91dd5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63de41f4348243998029f7d07aa5dc40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:56.512589 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860af1f4-328c-47ab-919e-d63a8fc91dd5 service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:56.512918 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860af1f4-328c-47ab-919e-d63a8fc91dd5 service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:56.513194 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860af1f4-328c-47ab-919e-d63a8fc91dd5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:56.515091 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-860af1f4-328c-47ab-919e-d63a8fc91dd5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:56.529206 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458344ae-7f5e-47e0-bf2f-12184b35a06e admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:56.532988 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4e9e82-6c54-4b5b-9bd0-d672fec8e4e2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c163b39e61648c385f065ce8091c3ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:56.533807 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4e9e82-6c54-4b5b-9bd0-d672fec8e4e2 service glance] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:56.534110 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4e9e82-6c54-4b5b-9bd0-d672fec8e4e2 service glance] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:56.534552 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4e9e82-6c54-4b5b-9bd0-d672fec8e4e2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:56.535858 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:25:56.536358 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-fa4e9e82-6c54-4b5b-9bd0-d672fec8e4e2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:56.552687 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4e9e82-6c54-4b5b-9bd0-d672fec8e4e2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:56.554187 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4e9e82-6c54-4b5b-9bd0-d672fec8e4e2 service glance] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:56.566293 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1284/2581] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:25:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:25:56.567733 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b999d377-ca6c-4cdc-a754-eb62b1bdd33b None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:56.568026 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b999d377-ca6c-4cdc-a754-eb62b1bdd33b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:56.568416 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b999d377-ca6c-4cdc-a754-eb62b1bdd33b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:56.591091 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-b999d377-ca6c-4cdc-a754-eb62b1bdd33b None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:25:56.592426 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b999d377-ca6c-4cdc-a754-eb62b1bdd33b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:56.604785 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860af1f4-328c-47ab-919e-d63a8fc91dd5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:56.606416 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860af1f4-328c-47ab-919e-d63a8fc91dd5 service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:56.616184 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1298/2582] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:25:56 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:25:56.617100 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860af1f4-328c-47ab-919e-d63a8fc91dd5 service nova] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:56.628598 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860af1f4-328c-47ab-919e-d63a8fc91dd5 service nova] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:56.632858 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b16042-d937-41a8-93b0-a5e10e2cf645 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c163b39e61648c385f065ce8091c3ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:56.633536 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b16042-d937-41a8-93b0-a5e10e2cf645 service glance] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:56.633857 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b16042-d937-41a8-93b0-a5e10e2cf645 service glance] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:56.634164 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b16042-d937-41a8-93b0-a5e10e2cf645 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:56.644893 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b16042-d937-41a8-93b0-a5e10e2cf645 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:56.646517 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b16042-d937-41a8-93b0-a5e10e2cf645 service glance] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:56.658520 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1299/2583] 10.222.0.43 () {60 vars in 1396 bytes} [Tue Sep 6 16:25:56 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3264 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:25:56.660623 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1285/2584] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:25:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:25:56.668254 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abda564-f787-42e3-88ce-fc02189279a8 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:56.668541 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abda564-f787-42e3-88ce-fc02189279a8 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:56.668827 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abda564-f787-42e3-88ce-fc02189279a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:56.695671 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-0abda564-f787-42e3-88ce-fc02189279a8 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:25:56.697100 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0abda564-f787-42e3-88ce-fc02189279a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:56.705139 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1300/2585] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:25:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:25:56.710663 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b999d377-ca6c-4cdc-a754-eb62b1bdd33b None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:56.724069 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b999d377-ca6c-4cdc-a754-eb62b1bdd33b None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:56.730053 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8005f19d-9e95-4daa-84c8-e148b9050f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:56.730744 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8005f19d-9e95-4daa-84c8-e148b9050f0b admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:56.731122 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8005f19d-9e95-4daa-84c8-e148b9050f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:56.731400 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8005f19d-9e95-4daa-84c8-e148b9050f0b admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:56.732142 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8005f19d-9e95-4daa-84c8-e148b9050f0b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:25:56.732486 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8005f19d-9e95-4daa-84c8-e148b9050f0b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:56.734438 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8005f19d-9e95-4daa-84c8-e148b9050f0b admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:56.734678 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abda564-f787-42e3-88ce-fc02189279a8 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:56.736665 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0abda564-f787-42e3-88ce-fc02189279a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:56.748812 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1286/2586] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 6 16:25:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:25:56.752992 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8005f19d-9e95-4daa-84c8-e148b9050f0b admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:56.763710 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8005f19d-9e95-4daa-84c8-e148b9050f0b admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:56.773438 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f32efeb-b216-4b15-b39a-7b07d79d5f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:56.774467 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f32efeb-b216-4b15-b39a-7b07d79d5f1d admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:56.774752 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f32efeb-b216-4b15-b39a-7b07d79d5f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:56.775399 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f32efeb-b216-4b15-b39a-7b07d79d5f1d admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:56.776470 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f32efeb-b216-4b15-b39a-7b07d79d5f1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:56.778675 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f32efeb-b216-4b15-b39a-7b07d79d5f1d admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:56.785474 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:25:56.808735 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abda564-f787-42e3-88ce-fc02189279a8 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:56.821203 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2200ef-a7ac-4fa9-a586-965642054b51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:56.821860 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2200ef-a7ac-4fa9-a586-965642054b51 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:56.822139 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2200ef-a7ac-4fa9-a586-965642054b51 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:56.822406 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2200ef-a7ac-4fa9-a586-965642054b51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:56.823673 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1287/2587] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:25:56 2022] POST /identity/v3/projects => generated 365 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:25:56.828319 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-de2200ef-a7ac-4fa9-a586-965642054b51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:56.828533 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f32efeb-b216-4b15-b39a-7b07d79d5f1d admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:56.838422 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2200ef-a7ac-4fa9-a586-965642054b51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:56.839974 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2200ef-a7ac-4fa9-a586-965642054b51 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:56.848315 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1301/2588] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:25:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:25:56.901090 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f32efeb-b216-4b15-b39a-7b07d79d5f1d admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:56.906509 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5f0b4d-264a-42fb-8734-ba70b9541c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:56.907202 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5f0b4d-264a-42fb-8734-ba70b9541c72 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:56.907510 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5f0b4d-264a-42fb-8734-ba70b9541c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:56.907792 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5f0b4d-264a-42fb-8734-ba70b9541c72 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:56.908545 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5f0b4d-264a-42fb-8734-ba70b9541c72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:56.910799 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5f0b4d-264a-42fb-8734-ba70b9541c72 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:56.926566 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2200ef-a7ac-4fa9-a586-965642054b51 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:56.928464 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-de2200ef-a7ac-4fa9-a586-965642054b51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:56.935775 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1288/2589] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:25:56 2022] POST /identity/v3/users => generated 283 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:25:56.939273 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2200ef-a7ac-4fa9-a586-965642054b51 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:56.940185 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5f0b4d-264a-42fb-8734-ba70b9541c72 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:56.951121 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5f0b4d-264a-42fb-8734-ba70b9541c72 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:56.956081 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f9f7d6-fc20-48dc-b209-47bf5ca69cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:56.956692 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f9f7d6-fc20-48dc-b209-47bf5ca69cda admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:56.957005 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f9f7d6-fc20-48dc-b209-47bf5ca69cda admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:56.957277 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f9f7d6-fc20-48dc-b209-47bf5ca69cda admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:56.957926 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f9f7d6-fc20-48dc-b209-47bf5ca69cda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:56.959540 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f9f7d6-fc20-48dc-b209-47bf5ca69cda admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:56.972800 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1289/2590] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:25:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:25:56.976158 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f9f7d6-fc20-48dc-b209-47bf5ca69cda admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:56.987238 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f9f7d6-fc20-48dc-b209-47bf5ca69cda admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:56.992913 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9f3c98-5e54-46f0-ba4c-8cf4c83fd0c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:56.993590 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9f3c98-5e54-46f0-ba4c-8cf4c83fd0c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:56.993877 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9f3c98-5e54-46f0-ba4c-8cf4c83fd0c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:56.994265 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9f3c98-5e54-46f0-ba4c-8cf4c83fd0c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:56.994913 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9f3c98-5e54-46f0-ba4c-8cf4c83fd0c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:56.996195 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f053fc-9cda-4384-8c76-9b93fdd69a2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:56.996722 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9f3c98-5e54-46f0-ba4c-8cf4c83fd0c0 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:56.996914 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f053fc-9cda-4384-8c76-9b93fdd69a2d service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:56.997137 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f053fc-9cda-4384-8c76-9b93fdd69a2d service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:56.997400 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f053fc-9cda-4384-8c76-9b93fdd69a2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:56.999637 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e6f053fc-9cda-4384-8c76-9b93fdd69a2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:57.008268 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1290/2591] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:25:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:25:57.012893 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9f3c98-5e54-46f0-ba4c-8cf4c83fd0c0 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:57.023696 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9f3c98-5e54-46f0-ba4c-8cf4c83fd0c0 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:57.028923 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aaa3b4-26e0-4e0e-8aa3-525cfe20b0fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:57.029604 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aaa3b4-26e0-4e0e-8aa3-525cfe20b0fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:57.029955 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aaa3b4-26e0-4e0e-8aa3-525cfe20b0fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:57.030225 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aaa3b4-26e0-4e0e-8aa3-525cfe20b0fa admin admin] PATH_INFO: `/v3/projects/6338ffd917224879a6dccfda2fcc03b4/users/e1349c885a46450588ccd90d79a04d9c/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:57.043398 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:25:57.047499 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aaa3b4-26e0-4e0e-8aa3-525cfe20b0fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=6338ffd917224879a6dccfda2fcc03b4, user_id=e1349c885a46450588ccd90d79a04d9c, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:57.050269 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aaa3b4-26e0-4e0e-8aa3-525cfe20b0fa admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:57.064997 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1291/2592] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:25:57 2022] PUT /identity/v3/projects/6338ffd917224879a6dccfda2fcc03b4/users/e1349c885a46450588ccd90d79a04d9c/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:25:57.075822 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f48aeb-f7ad-4264-a0c5-507ae4a2d167 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:57.076071 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f48aeb-f7ad-4264-a0c5-507ae4a2d167 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:57.076326 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f48aeb-f7ad-4264-a0c5-507ae4a2d167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:57.085934 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f053fc-9cda-4384-8c76-9b93fdd69a2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:57.087662 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f053fc-9cda-4384-8c76-9b93fdd69a2d service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:57.100599 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1302/2593] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:25:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:25:57.106594 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-53f48aeb-f7ad-4264-a0c5-507ae4a2d167 None None] MFA Rules not processed for user `e1349c885a46450588ccd90d79a04d9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:25:57.107909 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-53f48aeb-f7ad-4264-a0c5-507ae4a2d167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:57.177964 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1292/2594] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:25:57 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:25:57.190517 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f053fc-9cda-4384-8c76-9b93fdd69a2d service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:57.249272 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f053fc-9cda-4384-8c76-9b93fdd69a2d service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:57.305146 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e7460c-c177-40db-b88e-a501c7d3ec69 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c163b39e61648c385f065ce8091c3ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:57.305761 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e7460c-c177-40db-b88e-a501c7d3ec69 service glance] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:57.306041 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e7460c-c177-40db-b88e-a501c7d3ec69 service glance] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:57.306310 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e7460c-c177-40db-b88e-a501c7d3ec69 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:57.315042 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e7460c-c177-40db-b88e-a501c7d3ec69 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:57.316476 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e7460c-c177-40db-b88e-a501c7d3ec69 service glance] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:57.325278 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1303/2595] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:25:57 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:25:59.154757 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ea951a-0091-4ddf-9c48-d2f34bdff570 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:59.155297 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ea951a-0091-4ddf-9c48-d2f34bdff570 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:59.155678 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ea951a-0091-4ddf-9c48-d2f34bdff570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:59.177395 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-c5ea951a-0091-4ddf-9c48-d2f34bdff570 None None] MFA Rules not processed for user `d865006413b24ec3922619311f81f353`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:25:59.178666 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c5ea951a-0091-4ddf-9c48-d2f34bdff570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:59.193783 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:25:59.242783 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1293/2596] 10.222.0.43 () {58 vars in 1009 bytes} [Tue Sep 6 16:25:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:25:59.264768 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e7460c-c177-40db-b88e-a501c7d3ec69 service glance] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:59.266390 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d7e7460c-c177-40db-b88e-a501c7d3ec69 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:59.345044 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e7460c-c177-40db-b88e-a501c7d3ec69 service glance] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:59.350514 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfba44f-b4ea-4323-90a1-250de7d34929 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '312026c080b64aff9dd483d3591288a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:59.351182 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfba44f-b4ea-4323-90a1-250de7d34929 service swift] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:59.351531 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfba44f-b4ea-4323-90a1-250de7d34929 service swift] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:59.351808 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfba44f-b4ea-4323-90a1-250de7d34929 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:59.361673 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfba44f-b4ea-4323-90a1-250de7d34929 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:59.363512 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfba44f-b4ea-4323-90a1-250de7d34929 service swift] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:59.373733 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1304/2597] 10.222.0.43 () {60 vars in 1384 bytes} [Tue Sep 6 16:25:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 16:26:09.233243 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=16 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:26:09.369377 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=95 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:26:31.272612 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9600c4fb-3379-427e-9a03-ad18954b0e22 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:26:31.273202 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9600c4fb-3379-427e-9a03-ad18954b0e22 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:26:31.273311 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9600c4fb-3379-427e-9a03-ad18954b0e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:26:31.324482 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-9600c4fb-3379-427e-9a03-ad18954b0e22 None None] MFA Rules not processed for user `d865006413b24ec3922619311f81f353`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:26:31.325773 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-9600c4fb-3379-427e-9a03-ad18954b0e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:26:31.348641 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1294/2598] 10.222.0.43 () {58 vars in 967 bytes} [Tue Sep 6 16:26:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:26:31.361437 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfba44f-b4ea-4323-90a1-250de7d34929 service swift] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:26:31.382907 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfba44f-b4ea-4323-90a1-250de7d34929 service swift] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:26:31.388571 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c16a61-ccc7-4280-9420-6082fe1b05a2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '312026c080b64aff9dd483d3591288a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:26:31.389222 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c16a61-ccc7-4280-9420-6082fe1b05a2 service swift] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:26:31.389502 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c16a61-ccc7-4280-9420-6082fe1b05a2 service swift] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:26:31.389773 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c16a61-ccc7-4280-9420-6082fe1b05a2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:26:31.398673 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c16a61-ccc7-4280-9420-6082fe1b05a2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:26:31.400621 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c16a61-ccc7-4280-9420-6082fe1b05a2 service swift] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:26:31.409509 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1305/2599] 10.222.0.43 () {60 vars in 1384 bytes} [Tue Sep 6 16:26:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 16:26:41.345802 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=18 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:26:41.405861 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=10 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:26:46.946329 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9600c4fb-3379-427e-9a03-ad18954b0e22 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:26:47.024543 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9600c4fb-3379-427e-9a03-ad18954b0e22 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:26:47.032012 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c71eeb-8376-4f85-9556-8f4cdc306f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:26:47.033122 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c71eeb-8376-4f85-9556-8f4cdc306f1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:26:47.033596 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c71eeb-8376-4f85-9556-8f4cdc306f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:26:47.034044 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c71eeb-8376-4f85-9556-8f4cdc306f1c admin admin] PATH_INFO: `/v3/users/e1349c885a46450588ccd90d79a04d9c` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:26:47.035859 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c71eeb-8376-4f85-9556-8f4cdc306f1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1349c885a46450588ccd90d79a04d9c)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:26:47.039558 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c71eeb-8376-4f85-9556-8f4cdc306f1c admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:26:47.098283 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d6c71eeb-8376-4f85-9556-8f4cdc306f1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1349c885a46450588ccd90d79a04d9c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:26:47.099198 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d6c71eeb-8376-4f85-9556-8f4cdc306f1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1349c885a46450588ccd90d79a04d9c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:26:47.106343 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d6c71eeb-8376-4f85-9556-8f4cdc306f1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1349c885a46450588ccd90d79a04d9c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:26:47.118706 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d6c71eeb-8376-4f85-9556-8f4cdc306f1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1349c885a46450588ccd90d79a04d9c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:26:47.126451 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d6c71eeb-8376-4f85-9556-8f4cdc306f1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1349c885a46450588ccd90d79a04d9c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:26:47.132457 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1295/2600] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:26:46 2022] DELETE /identity/v3/users/e1349c885a46450588ccd90d79a04d9c => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:26:47.137568 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b836815-5e57-4756-af02-6f547e8cbc84 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:26:47.138209 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b836815-5e57-4756-af02-6f547e8cbc84 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:26:47.138209 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b836815-5e57-4756-af02-6f547e8cbc84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:26:47.158351 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-7b836815-5e57-4756-af02-6f547e8cbc84 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:26:47.159547 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-7b836815-5e57-4756-af02-6f547e8cbc84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:26:47.218307 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1306/2601] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:26:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:26:47.228432 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c71eeb-8376-4f85-9556-8f4cdc306f1c admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:26:47.230509 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d6c71eeb-8376-4f85-9556-8f4cdc306f1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:26:47.295433 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c71eeb-8376-4f85-9556-8f4cdc306f1c admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:26:47.299379 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd3401c-6183-4a50-99d2-091c7879bd0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:26:47.300012 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3401c-6183-4a50-99d2-091c7879bd0b service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:26:47.300287 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3401c-6183-4a50-99d2-091c7879bd0b service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:26:47.300555 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3401c-6183-4a50-99d2-091c7879bd0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:26:47.311168 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd3401c-6183-4a50-99d2-091c7879bd0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:26:47.312660 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd3401c-6183-4a50-99d2-091c7879bd0b service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:26:47.322563 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1296/2602] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:26:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:26:47.601317 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b836815-5e57-4756-af02-6f547e8cbc84 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:26:47.602884 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-7b836815-5e57-4756-af02-6f547e8cbc84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:26:47.615342 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b836815-5e57-4756-af02-6f547e8cbc84 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:26:47.621593 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4543077-837f-4bd2-9770-c682df4022af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:26:47.622407 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4543077-837f-4bd2-9770-c682df4022af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:26:47.622690 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4543077-837f-4bd2-9770-c682df4022af admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:26:47.622962 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4543077-837f-4bd2-9770-c682df4022af admin admin] PATH_INFO: `/v3/projects/6338ffd917224879a6dccfda2fcc03b4` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:26:47.624158 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4543077-837f-4bd2-9770-c682df4022af admin admin] RBAC: Authorizing `identity:delete_project(project_id=6338ffd917224879a6dccfda2fcc03b4)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:26:47.627321 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4543077-837f-4bd2-9770-c682df4022af admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:26:47.679808 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b4543077-837f-4bd2-9770-c682df4022af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6338ffd917224879a6dccfda2fcc03b4', 'request_id': 'req-b4543077-837f-4bd2-9770-c682df4022af'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:26:47.685390 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b4543077-837f-4bd2-9770-c682df4022af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6338ffd917224879a6dccfda2fcc03b4', 'request_id': 'req-b4543077-837f-4bd2-9770-c682df4022af'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:26:47.693769 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b4543077-837f-4bd2-9770-c682df4022af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6338ffd917224879a6dccfda2fcc03b4', 'request_id': 'req-b4543077-837f-4bd2-9770-c682df4022af'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:26:47.705874 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b4543077-837f-4bd2-9770-c682df4022af admin admin] The token cache is being invalidate because project 6338ffd917224879a6dccfda2fcc03b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:26:47.706166 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b4543077-837f-4bd2-9770-c682df4022af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:26:47.706822 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b4543077-837f-4bd2-9770-c682df4022af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:26:47.708294 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1307/2603] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:26:47 2022] DELETE /identity/v3/projects/6338ffd917224879a6dccfda2fcc03b4 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:26:56.845756 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd3401c-6183-4a50-99d2-091c7879bd0b service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:26:56.847440 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7dd3401c-6183-4a50-99d2-091c7879bd0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:26:56.876814 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:26:56.931163 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd3401c-6183-4a50-99d2-091c7879bd0b service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:26:56.995011 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ca6722-f078-4ab1-94ad-65ff5a1cf70b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63de41f4348243998029f7d07aa5dc40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:26:56.995616 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ca6722-f078-4ab1-94ad-65ff5a1cf70b service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:26:56.995889 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ca6722-f078-4ab1-94ad-65ff5a1cf70b service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:26:56.996156 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ca6722-f078-4ab1-94ad-65ff5a1cf70b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:26:56.998039 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f7ca6722-f078-4ab1-94ad-65ff5a1cf70b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:26:57.081271 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ca6722-f078-4ab1-94ad-65ff5a1cf70b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:26:57.083006 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ca6722-f078-4ab1-94ad-65ff5a1cf70b service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:26:57.102506 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1297/2604] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:26:56 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:26:57.702588 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:26:58.529901 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4543077-837f-4bd2-9770-c682df4022af admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:26:58.531480 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b4543077-837f-4bd2-9770-c682df4022af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:26:58.601801 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4543077-837f-4bd2-9770-c682df4022af admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:26:58.607686 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078bbdcf-bea4-45f6-b3e5-cc39ee71c974 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:26:58.608597 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078bbdcf-bea4-45f6-b3e5-cc39ee71c974 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:26:58.609009 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078bbdcf-bea4-45f6-b3e5-cc39ee71c974 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:26:58.609390 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078bbdcf-bea4-45f6-b3e5-cc39ee71c974 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:26:58.611810 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-078bbdcf-bea4-45f6-b3e5-cc39ee71c974 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:26:58.629627 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078bbdcf-bea4-45f6-b3e5-cc39ee71c974 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:26:58.631059 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078bbdcf-bea4-45f6-b3e5-cc39ee71c974 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:26:58.641438 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1308/2605] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:26:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:27:07.099579 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=87 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:27:08.636201 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=26 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:27:23.513570 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ca6722-f078-4ab1-94ad-65ff5a1cf70b service nova] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:23.515222 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f7ca6722-f078-4ab1-94ad-65ff5a1cf70b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:23.529608 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ca6722-f078-4ab1-94ad-65ff5a1cf70b service nova] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:23.535159 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a26f979-f32e-4399-a37b-385cfcfc7d1b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63de41f4348243998029f7d07aa5dc40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:23.535771 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a26f979-f32e-4399-a37b-385cfcfc7d1b service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:23.536041 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a26f979-f32e-4399-a37b-385cfcfc7d1b service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:23.536309 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a26f979-f32e-4399-a37b-385cfcfc7d1b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:23.538490 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7a26f979-f32e-4399-a37b-385cfcfc7d1b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:23.554469 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a26f979-f32e-4399-a37b-385cfcfc7d1b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:23.556037 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a26f979-f32e-4399-a37b-385cfcfc7d1b service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:23.568249 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1298/2606] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:27:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:27:29.501134 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078bbdcf-bea4-45f6-b3e5-cc39ee71c974 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:29.514337 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078bbdcf-bea4-45f6-b3e5-cc39ee71c974 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:29.520185 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fdd5cc-f78f-47c3-84e0-df582ff57969 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:29.520793 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fdd5cc-f78f-47c3-84e0-df582ff57969 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:29.521090 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fdd5cc-f78f-47c3-84e0-df582ff57969 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:29.521357 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fdd5cc-f78f-47c3-84e0-df582ff57969 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:29.523441 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-30fdd5cc-f78f-47c3-84e0-df582ff57969 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:29.597884 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fdd5cc-f78f-47c3-84e0-df582ff57969 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:29.599595 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fdd5cc-f78f-47c3-84e0-df582ff57969 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:29.610367 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1309/2607] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:27:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:27:32.622291 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a26f979-f32e-4399-a37b-385cfcfc7d1b service nova] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:32.623882 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7a26f979-f32e-4399-a37b-385cfcfc7d1b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:32.641057 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a26f979-f32e-4399-a37b-385cfcfc7d1b service nova] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:32.645860 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8dcc07-89c9-4df1-8e7d-1518295bfbb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:32.646511 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8dcc07-89c9-4df1-8e7d-1518295bfbb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:32.646788 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8dcc07-89c9-4df1-8e7d-1518295bfbb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:32.647061 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8dcc07-89c9-4df1-8e7d-1518295bfbb1 admin admin] PATH_INFO: `/v3/users/e225aec64aee47449542aecbec0ae561` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:32.648204 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8dcc07-89c9-4df1-8e7d-1518295bfbb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e225aec64aee47449542aecbec0ae561)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:32.650180 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8dcc07-89c9-4df1-8e7d-1518295bfbb1 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:32.705084 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3d8dcc07-89c9-4df1-8e7d-1518295bfbb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e225aec64aee47449542aecbec0ae561'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:32.705772 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3d8dcc07-89c9-4df1-8e7d-1518295bfbb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e225aec64aee47449542aecbec0ae561'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:32.713243 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3d8dcc07-89c9-4df1-8e7d-1518295bfbb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e225aec64aee47449542aecbec0ae561'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:32.724879 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3d8dcc07-89c9-4df1-8e7d-1518295bfbb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e225aec64aee47449542aecbec0ae561'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:32.733551 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3d8dcc07-89c9-4df1-8e7d-1518295bfbb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e225aec64aee47449542aecbec0ae561'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:32.740561 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1299/2608] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:27:32 2022] DELETE /identity/v3/users/e225aec64aee47449542aecbec0ae561 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:27:32.936742 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fdd5cc-f78f-47c3-84e0-df582ff57969 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:32.947939 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fdd5cc-f78f-47c3-84e0-df582ff57969 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:33.011715 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97be34a9-bae7-4c4f-91ad-d150acc41a65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:33.012425 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97be34a9-bae7-4c4f-91ad-d150acc41a65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:33.012785 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97be34a9-bae7-4c4f-91ad-d150acc41a65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:33.013127 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97be34a9-bae7-4c4f-91ad-d150acc41a65 admin admin] PATH_INFO: `/v3/projects/ad0bef00a05c4029a009992a7f96fa80` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:33.014490 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97be34a9-bae7-4c4f-91ad-d150acc41a65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad0bef00a05c4029a009992a7f96fa80)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:33.016540 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97be34a9-bae7-4c4f-91ad-d150acc41a65 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:33.064922 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-97be34a9-bae7-4c4f-91ad-d150acc41a65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad0bef00a05c4029a009992a7f96fa80', 'request_id': 'req-97be34a9-bae7-4c4f-91ad-d150acc41a65'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:33.070294 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-97be34a9-bae7-4c4f-91ad-d150acc41a65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad0bef00a05c4029a009992a7f96fa80', 'request_id': 'req-97be34a9-bae7-4c4f-91ad-d150acc41a65'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:33.078759 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-97be34a9-bae7-4c4f-91ad-d150acc41a65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad0bef00a05c4029a009992a7f96fa80', 'request_id': 'req-97be34a9-bae7-4c4f-91ad-d150acc41a65'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:33.091058 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-97be34a9-bae7-4c4f-91ad-d150acc41a65 admin admin] The token cache is being invalidate because project ad0bef00a05c4029a009992a7f96fa80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:27:33.091490 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-97be34a9-bae7-4c4f-91ad-d150acc41a65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:33.092249 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-97be34a9-bae7-4c4f-91ad-d150acc41a65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:33.094073 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1310/2609] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:27:32 2022] DELETE /identity/v3/projects/ad0bef00a05c4029a009992a7f96fa80 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:27:33.101982 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905d1d3d-1ae3-4915-84b6-7a6d46d3ef40 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:33.102271 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905d1d3d-1ae3-4915-84b6-7a6d46d3ef40 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:33.102529 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905d1d3d-1ae3-4915-84b6-7a6d46d3ef40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:33.124399 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-905d1d3d-1ae3-4915-84b6-7a6d46d3ef40 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:27:33.125663 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-905d1d3d-1ae3-4915-84b6-7a6d46d3ef40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:33.187328 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1300/2610] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:27:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:27:33.200938 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cacf6c2-8b16-4b9b-93ac-0d8792929a26 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:33.201221 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cacf6c2-8b16-4b9b-93ac-0d8792929a26 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:33.201487 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cacf6c2-8b16-4b9b-93ac-0d8792929a26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:33.224814 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-7cacf6c2-8b16-4b9b-93ac-0d8792929a26 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:27:33.226529 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-7cacf6c2-8b16-4b9b-93ac-0d8792929a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:33.238942 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1311/2611] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:27:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:27:33.243730 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905d1d3d-1ae3-4915-84b6-7a6d46d3ef40 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:33.254111 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905d1d3d-1ae3-4915-84b6-7a6d46d3ef40 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:33.259927 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7b26bb-bf3a-452b-a451-e7607784cc40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:33.260540 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b26bb-bf3a-452b-a451-e7607784cc40 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:33.260840 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b26bb-bf3a-452b-a451-e7607784cc40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:33.261100 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b26bb-bf3a-452b-a451-e7607784cc40 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:33.261743 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7b26bb-bf3a-452b-a451-e7607784cc40 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:27:33.262042 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7b26bb-bf3a-452b-a451-e7607784cc40 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:33.263680 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7b26bb-bf3a-452b-a451-e7607784cc40 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:33.277738 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1301/2612] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 6 16:27:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:27:33.281222 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cacf6c2-8b16-4b9b-93ac-0d8792929a26 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:33.291252 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cacf6c2-8b16-4b9b-93ac-0d8792929a26 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:33.297107 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06018150-5b70-4b1c-9321-2bb320d007cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:33.297715 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06018150-5b70-4b1c-9321-2bb320d007cb admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:33.297992 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06018150-5b70-4b1c-9321-2bb320d007cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:33.298259 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06018150-5b70-4b1c-9321-2bb320d007cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:33.299022 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06018150-5b70-4b1c-9321-2bb320d007cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:33.301233 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06018150-5b70-4b1c-9321-2bb320d007cb admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:33.319353 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:27:33.335611 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1312/2613] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:27:33 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:27:33.340416 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7b26bb-bf3a-452b-a451-e7607784cc40 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:33.356508 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7b26bb-bf3a-452b-a451-e7607784cc40 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:33.414508 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:27:33.432977 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b8fc1b-a1c7-4346-bfd0-9ea04f902ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:33.433874 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b8fc1b-a1c7-4346-bfd0-9ea04f902ab2 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:33.434241 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b8fc1b-a1c7-4346-bfd0-9ea04f902ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:33.434620 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b8fc1b-a1c7-4346-bfd0-9ea04f902ab2 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:33.435597 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b8fc1b-a1c7-4346-bfd0-9ea04f902ab2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:33.438580 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b8fc1b-a1c7-4346-bfd0-9ea04f902ab2 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:33.462555 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1302/2614] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:27:33 2022] POST /identity/v3/users => generated 293 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:27:33.466882 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06018150-5b70-4b1c-9321-2bb320d007cb admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:33.477834 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06018150-5b70-4b1c-9321-2bb320d007cb admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:33.484592 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8210cc21-5072-4f39-9983-ab1c1dc95782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:33.485563 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8210cc21-5072-4f39-9983-ab1c1dc95782 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:33.486019 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8210cc21-5072-4f39-9983-ab1c1dc95782 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:33.486533 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8210cc21-5072-4f39-9983-ab1c1dc95782 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:33.487623 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8210cc21-5072-4f39-9983-ab1c1dc95782 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:33.490426 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8210cc21-5072-4f39-9983-ab1c1dc95782 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:33.503926 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1313/2615] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:27:33 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:27:33.509558 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b8fc1b-a1c7-4346-bfd0-9ea04f902ab2 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:33.519967 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b8fc1b-a1c7-4346-bfd0-9ea04f902ab2 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:33.525573 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b17d73b-e67e-4d0f-9fdb-3979bdf15440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:33.526166 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b17d73b-e67e-4d0f-9fdb-3979bdf15440 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:33.526442 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b17d73b-e67e-4d0f-9fdb-3979bdf15440 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:33.526706 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b17d73b-e67e-4d0f-9fdb-3979bdf15440 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:33.527344 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b17d73b-e67e-4d0f-9fdb-3979bdf15440 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:33.528964 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b17d73b-e67e-4d0f-9fdb-3979bdf15440 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:33.539373 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1303/2616] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:27:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:27:33.544233 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8210cc21-5072-4f39-9983-ab1c1dc95782 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:33.559765 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8210cc21-5072-4f39-9983-ab1c1dc95782 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:33.566340 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3534b635-0bc8-4394-b9a7-b9ca0b681e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:33.566987 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3534b635-0bc8-4394-b9a7-b9ca0b681e27 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:33.567265 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3534b635-0bc8-4394-b9a7-b9ca0b681e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:33.567539 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3534b635-0bc8-4394-b9a7-b9ca0b681e27 admin admin] PATH_INFO: `/v3/projects/5c722f58536f492aa4eb96085452def4/users/523faa1ac64e4044b9fb651ec28249d9/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:33.583402 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3534b635-0bc8-4394-b9a7-b9ca0b681e27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c722f58536f492aa4eb96085452def4, user_id=523faa1ac64e4044b9fb651ec28249d9, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:33.586046 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3534b635-0bc8-4394-b9a7-b9ca0b681e27 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:33.600417 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1314/2617] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:27:33 2022] PUT /identity/v3/projects/5c722f58536f492aa4eb96085452def4/users/523faa1ac64e4044b9fb651ec28249d9/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:27:33.607520 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ce8858-2fce-4193-8e0b-c50a61a628d2 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:33.607854 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ce8858-2fce-4193-8e0b-c50a61a628d2 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:33.608221 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ce8858-2fce-4193-8e0b-c50a61a628d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:33.633302 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-a7ce8858-2fce-4193-8e0b-c50a61a628d2 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:27:33.634535 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a7ce8858-2fce-4193-8e0b-c50a61a628d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:33.698072 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1304/2618] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:27:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:27:33.707995 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3534b635-0bc8-4394-b9a7-b9ca0b681e27 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:33.710103 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3534b635-0bc8-4394-b9a7-b9ca0b681e27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:33.779193 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3534b635-0bc8-4394-b9a7-b9ca0b681e27 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:33.783830 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e207de33-8da9-4a6e-b526-af52d6dff0f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:33.784459 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e207de33-8da9-4a6e-b526-af52d6dff0f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:33.784735 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e207de33-8da9-4a6e-b526-af52d6dff0f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:33.785046 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e207de33-8da9-4a6e-b526-af52d6dff0f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:33.795187 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e207de33-8da9-4a6e-b526-af52d6dff0f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:33.796772 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e207de33-8da9-4a6e-b526-af52d6dff0f4 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:33.807155 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1315/2619] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:27:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:27:38.797092 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952ef4ec-854f-41f1-9e5e-6dd4dd3c9319 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:38.797570 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952ef4ec-854f-41f1-9e5e-6dd4dd3c9319 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:38.797655 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952ef4ec-854f-41f1-9e5e-6dd4dd3c9319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:38.829913 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-952ef4ec-854f-41f1-9e5e-6dd4dd3c9319 None None] MFA Rules not processed for user `523faa1ac64e4044b9fb651ec28249d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:27:38.831182 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-952ef4ec-854f-41f1-9e5e-6dd4dd3c9319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:38.901694 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1305/2620] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:27:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:27:38.917109 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e207de33-8da9-4a6e-b526-af52d6dff0f4 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:38.918818 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e207de33-8da9-4a6e-b526-af52d6dff0f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:39.011512 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e207de33-8da9-4a6e-b526-af52d6dff0f4 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:39.017735 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4def23c5-e4ef-47da-9715-5c58ef78fe82 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63de41f4348243998029f7d07aa5dc40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:39.018478 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4def23c5-e4ef-47da-9715-5c58ef78fe82 service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:39.018729 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4def23c5-e4ef-47da-9715-5c58ef78fe82 service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:39.019063 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4def23c5-e4ef-47da-9715-5c58ef78fe82 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:39.027744 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4def23c5-e4ef-47da-9715-5c58ef78fe82 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:39.029425 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4def23c5-e4ef-47da-9715-5c58ef78fe82 service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:39.039235 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1316/2621] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:27:38 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:27:40.610135 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3c6a46-49b4-4916-bf6c-818dd826e153 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:40.611092 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3c6a46-49b4-4916-bf6c-818dd826e153 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:40.611529 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3c6a46-49b4-4916-bf6c-818dd826e153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:40.639966 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-1c3c6a46-49b4-4916-bf6c-818dd826e153 None None] MFA Rules not processed for user `523faa1ac64e4044b9fb651ec28249d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:27:40.644002 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-1c3c6a46-49b4-4916-bf6c-818dd826e153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:40.658632 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1306/2622] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:27:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:27:40.813337 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7815c47b-0fbe-43cc-812d-830d0e04ff1f None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:40.813940 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7815c47b-0fbe-43cc-812d-830d0e04ff1f None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:40.814175 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7815c47b-0fbe-43cc-812d-830d0e04ff1f None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:40.817445 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1317/2623] 10.222.0.43 () {54 vars in 921 bytes} [Tue Sep 6 16:27:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:27:40.827111 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e2753-b2db-4704-9418-da5a748e6471 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:40.827636 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e2753-b2db-4704-9418-da5a748e6471 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:40.828126 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e2753-b2db-4704-9418-da5a748e6471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:40.886768 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-182e2753-b2db-4704-9418-da5a748e6471 None None] MFA Rules not processed for user `10565aefeaaa40549ed38bb9e0f4b189`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:27:40.887993 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-182e2753-b2db-4704-9418-da5a748e6471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:40.895425 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1307/2624] 10.222.0.43 () {58 vars in 1018 bytes} [Tue Sep 6 16:27:40 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:27:40.930347 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7815c47b-0fbe-43cc-812d-830d0e04ff1f None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:40.932471 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-7815c47b-0fbe-43cc-812d-830d0e04ff1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:40.973141 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7815c47b-0fbe-43cc-812d-830d0e04ff1f None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:40.985888 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29505b0e-b4ce-430f-9cff-da32a3357c08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:40.987092 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29505b0e-b4ce-430f-9cff-da32a3357c08 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:40.987578 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29505b0e-b4ce-430f-9cff-da32a3357c08 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:40.988721 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29505b0e-b4ce-430f-9cff-da32a3357c08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:41.000427 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29505b0e-b4ce-430f-9cff-da32a3357c08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:41.003730 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29505b0e-b4ce-430f-9cff-da32a3357c08 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:41.029795 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1318/2625] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:27:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:27:41.109706 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182e2753-b2db-4704-9418-da5a748e6471 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:41.122733 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182e2753-b2db-4704-9418-da5a748e6471 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:41.128359 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1b35c9-9937-4599-a0bc-775cb510ad68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:41.129007 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b35c9-9937-4599-a0bc-775cb510ad68 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:41.129282 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b35c9-9937-4599-a0bc-775cb510ad68 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:41.129561 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b35c9-9937-4599-a0bc-775cb510ad68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:41.131972 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-1a1b35c9-9937-4599-a0bc-775cb510ad68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:41.144308 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1b35c9-9937-4599-a0bc-775cb510ad68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:41.146001 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1b35c9-9937-4599-a0bc-775cb510ad68 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:41.155666 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1308/2626] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:27:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:27:41.185275 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7a5b99-c754-4e8e-b38d-5a3d171e2147 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:41.185707 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7a5b99-c754-4e8e-b38d-5a3d171e2147 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:41.186469 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7a5b99-c754-4e8e-b38d-5a3d171e2147 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:41.188265 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1319/2627] 10.222.0.43 () {54 vars in 921 bytes} [Tue Sep 6 16:27:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:27:41.195695 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d78d993-2d87-467b-90f7-02e29d07f5b7 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:41.195987 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d78d993-2d87-467b-90f7-02e29d07f5b7 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:41.196430 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d78d993-2d87-467b-90f7-02e29d07f5b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:41.235819 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:27:41.247489 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-6d78d993-2d87-467b-90f7-02e29d07f5b7 None None] MFA Rules not processed for user `63de41f4348243998029f7d07aa5dc40`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:27:41.251200 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-6d78d993-2d87-467b-90f7-02e29d07f5b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:41.283675 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1309/2628] 10.222.0.43 () {58 vars in 1018 bytes} [Tue Sep 6 16:27:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:27:41.297813 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7a5b99-c754-4e8e-b38d-5a3d171e2147 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:41.310247 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7a5b99-c754-4e8e-b38d-5a3d171e2147 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:41.314899 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f2050d-1ec4-4954-8768-782f40643dcd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:41.315723 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f2050d-1ec4-4954-8768-782f40643dcd service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:41.316019 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f2050d-1ec4-4954-8768-782f40643dcd service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:41.316427 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f2050d-1ec4-4954-8768-782f40643dcd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:41.334379 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f2050d-1ec4-4954-8768-782f40643dcd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:41.336247 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f2050d-1ec4-4954-8768-782f40643dcd service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:41.353842 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1320/2629] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:27:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:27:41.479340 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d78d993-2d87-467b-90f7-02e29d07f5b7 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:41.480829 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-6d78d993-2d87-467b-90f7-02e29d07f5b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:41.623812 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d78d993-2d87-467b-90f7-02e29d07f5b7 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:41.630845 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e62832-3709-4f12-bb98-1691fe30f6ff service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71c2912a44f04bd983b4d4112dbee734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:41.631708 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e62832-3709-4f12-bb98-1691fe30f6ff service placement] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:41.632322 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e62832-3709-4f12-bb98-1691fe30f6ff service placement] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:41.632707 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e62832-3709-4f12-bb98-1691fe30f6ff service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:41.637355 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f3e62832-3709-4f12-bb98-1691fe30f6ff service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:41.709782 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e62832-3709-4f12-bb98-1691fe30f6ff service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:41.712058 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e62832-3709-4f12-bb98-1691fe30f6ff service placement] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:41.741364 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1310/2630] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:27:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:27:46.038760 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f2050d-1ec4-4954-8768-782f40643dcd service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:46.052004 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f2050d-1ec4-4954-8768-782f40643dcd service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:46.057311 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2117521-cbea-446b-beb2-8da94ee28756 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:46.058452 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2117521-cbea-446b-beb2-8da94ee28756 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:46.058795 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2117521-cbea-446b-beb2-8da94ee28756 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:46.059137 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2117521-cbea-446b-beb2-8da94ee28756 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:46.061287 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f2117521-cbea-446b-beb2-8da94ee28756 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:46.075070 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2117521-cbea-446b-beb2-8da94ee28756 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:46.076679 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2117521-cbea-446b-beb2-8da94ee28756 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:46.089859 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1321/2631] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:27:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:27:50.367644 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e62832-3709-4f12-bb98-1691fe30f6ff service placement] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:50.369119 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f3e62832-3709-4f12-bb98-1691fe30f6ff service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:50.399395 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e62832-3709-4f12-bb98-1691fe30f6ff service placement] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:50.404248 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579bce36-1bf3-40a1-9483-a18665236155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:50.404890 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579bce36-1bf3-40a1-9483-a18665236155 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:50.405142 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579bce36-1bf3-40a1-9483-a18665236155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:50.405426 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579bce36-1bf3-40a1-9483-a18665236155 admin admin] PATH_INFO: `/v3/users/c3a65f26ab214feab60c7b6380539076` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:50.406612 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579bce36-1bf3-40a1-9483-a18665236155 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3a65f26ab214feab60c7b6380539076)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:50.408604 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579bce36-1bf3-40a1-9483-a18665236155 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:50.467274 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-579bce36-1bf3-40a1-9483-a18665236155 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3a65f26ab214feab60c7b6380539076'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:50.467934 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-579bce36-1bf3-40a1-9483-a18665236155 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3a65f26ab214feab60c7b6380539076'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:50.475669 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-579bce36-1bf3-40a1-9483-a18665236155 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3a65f26ab214feab60c7b6380539076'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:50.489679 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-579bce36-1bf3-40a1-9483-a18665236155 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3a65f26ab214feab60c7b6380539076'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:50.500042 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-579bce36-1bf3-40a1-9483-a18665236155 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3a65f26ab214feab60c7b6380539076'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:50.506569 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1311/2632] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:27:50 2022] DELETE /identity/v3/users/c3a65f26ab214feab60c7b6380539076 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:27:50.708693 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2117521-cbea-446b-beb2-8da94ee28756 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:50.725617 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2117521-cbea-446b-beb2-8da94ee28756 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:50.732208 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:27:50.805125 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fecaa5-1062-4084-81fa-8dddb4197017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:50.805975 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fecaa5-1062-4084-81fa-8dddb4197017 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:50.806249 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fecaa5-1062-4084-81fa-8dddb4197017 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:50.806550 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fecaa5-1062-4084-81fa-8dddb4197017 admin admin] PATH_INFO: `/v3/projects/8b93a6978c404ca4aa497919103b5283` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:50.807960 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fecaa5-1062-4084-81fa-8dddb4197017 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b93a6978c404ca4aa497919103b5283)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:50.810602 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fecaa5-1062-4084-81fa-8dddb4197017 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:50.858290 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-58fecaa5-1062-4084-81fa-8dddb4197017 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b93a6978c404ca4aa497919103b5283', 'request_id': 'req-58fecaa5-1062-4084-81fa-8dddb4197017'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:50.865006 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-58fecaa5-1062-4084-81fa-8dddb4197017 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b93a6978c404ca4aa497919103b5283', 'request_id': 'req-58fecaa5-1062-4084-81fa-8dddb4197017'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:50.875469 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-58fecaa5-1062-4084-81fa-8dddb4197017 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b93a6978c404ca4aa497919103b5283', 'request_id': 'req-58fecaa5-1062-4084-81fa-8dddb4197017'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:50.889621 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-58fecaa5-1062-4084-81fa-8dddb4197017 admin admin] The token cache is being invalidate because project 8b93a6978c404ca4aa497919103b5283 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:27:50.889927 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-58fecaa5-1062-4084-81fa-8dddb4197017 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:50.890685 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-58fecaa5-1062-4084-81fa-8dddb4197017 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:50.892277 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1322/2633] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:27:50 2022] DELETE /identity/v3/projects/8b93a6978c404ca4aa497919103b5283 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:27:50.897458 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679a1fdf-a881-4ff2-94be-8f9d2282e889 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:50.897744 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679a1fdf-a881-4ff2-94be-8f9d2282e889 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:50.898015 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679a1fdf-a881-4ff2-94be-8f9d2282e889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:50.918946 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-679a1fdf-a881-4ff2-94be-8f9d2282e889 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:27:50.920062 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-679a1fdf-a881-4ff2-94be-8f9d2282e889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:50.983903 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:27:50.995561 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1312/2634] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:27:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:27:51.004538 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcbb772-ddcf-4f24-9ce5-1133772cdbf7 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:51.004868 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcbb772-ddcf-4f24-9ce5-1133772cdbf7 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:51.005146 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcbb772-ddcf-4f24-9ce5-1133772cdbf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:51.025582 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-afcbb772-ddcf-4f24-9ce5-1133772cdbf7 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:27:51.026796 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-afcbb772-ddcf-4f24-9ce5-1133772cdbf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:51.034842 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1323/2635] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:27:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:27:51.039356 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679a1fdf-a881-4ff2-94be-8f9d2282e889 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:51.049632 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679a1fdf-a881-4ff2-94be-8f9d2282e889 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:51.056474 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156ff018-fa70-4615-91b7-a22d366a240f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:51.057119 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156ff018-fa70-4615-91b7-a22d366a240f admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:51.057391 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156ff018-fa70-4615-91b7-a22d366a240f admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:51.057643 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156ff018-fa70-4615-91b7-a22d366a240f admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:51.058306 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156ff018-fa70-4615-91b7-a22d366a240f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:27:51.058601 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156ff018-fa70-4615-91b7-a22d366a240f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:51.060254 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156ff018-fa70-4615-91b7-a22d366a240f admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:51.072720 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1313/2636] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 6 16:27:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:27:51.077273 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcbb772-ddcf-4f24-9ce5-1133772cdbf7 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:51.086811 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcbb772-ddcf-4f24-9ce5-1133772cdbf7 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:51.091574 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e941616-59cb-41e3-8075-cb1943e9d896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:51.092171 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e941616-59cb-41e3-8075-cb1943e9d896 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:51.092453 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e941616-59cb-41e3-8075-cb1943e9d896 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:51.092725 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e941616-59cb-41e3-8075-cb1943e9d896 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:51.093391 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e941616-59cb-41e3-8075-cb1943e9d896 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:51.095314 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e941616-59cb-41e3-8075-cb1943e9d896 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:51.126333 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1324/2637] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:27:51 2022] POST /identity/v3/projects => generated 405 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:27:51.130706 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156ff018-fa70-4615-91b7-a22d366a240f admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:51.143777 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156ff018-fa70-4615-91b7-a22d366a240f admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:51.214779 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3703b20-7948-4e22-8127-adbcb9b41023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:51.215391 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3703b20-7948-4e22-8127-adbcb9b41023 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:51.215664 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3703b20-7948-4e22-8127-adbcb9b41023 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:51.215930 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3703b20-7948-4e22-8127-adbcb9b41023 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:51.216560 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3703b20-7948-4e22-8127-adbcb9b41023 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:51.218543 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3703b20-7948-4e22-8127-adbcb9b41023 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:51.242602 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1314/2638] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:27:51 2022] POST /identity/v3/users => generated 303 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:27:51.246989 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e941616-59cb-41e3-8075-cb1943e9d896 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:51.259936 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e941616-59cb-41e3-8075-cb1943e9d896 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:51.266134 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2677a4f8-53bb-4a78-b55c-b8f6296d3e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:51.266960 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2677a4f8-53bb-4a78-b55c-b8f6296d3e8a admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:51.267242 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2677a4f8-53bb-4a78-b55c-b8f6296d3e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:51.267759 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2677a4f8-53bb-4a78-b55c-b8f6296d3e8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:51.268599 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2677a4f8-53bb-4a78-b55c-b8f6296d3e8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:51.270572 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2677a4f8-53bb-4a78-b55c-b8f6296d3e8a admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:51.281454 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1325/2639] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:27:51 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:27:51.285735 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3703b20-7948-4e22-8127-adbcb9b41023 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:51.295816 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3703b20-7948-4e22-8127-adbcb9b41023 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:51.301160 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e27379-25d5-4504-aa81-38c46cff6d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:51.301740 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e27379-25d5-4504-aa81-38c46cff6d9b admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:51.302007 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e27379-25d5-4504-aa81-38c46cff6d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:51.302280 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e27379-25d5-4504-aa81-38c46cff6d9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:51.302928 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e27379-25d5-4504-aa81-38c46cff6d9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:51.304518 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e27379-25d5-4504-aa81-38c46cff6d9b admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:51.315972 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1315/2640] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:27:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:27:51.322594 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2677a4f8-53bb-4a78-b55c-b8f6296d3e8a admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:51.333591 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2677a4f8-53bb-4a78-b55c-b8f6296d3e8a admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:51.338349 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea29c124-b15c-4b61-a240-23f79aeb5e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:51.338998 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea29c124-b15c-4b61-a240-23f79aeb5e78 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:51.339272 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea29c124-b15c-4b61-a240-23f79aeb5e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:51.339537 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea29c124-b15c-4b61-a240-23f79aeb5e78 admin admin] PATH_INFO: `/v3/projects/33a1b243fddc47d386c9f6b5679af82b/users/ac07dedb10734d07921294f55cbf8be8/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:51.355246 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea29c124-b15c-4b61-a240-23f79aeb5e78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33a1b243fddc47d386c9f6b5679af82b, user_id=ac07dedb10734d07921294f55cbf8be8, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:51.358058 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea29c124-b15c-4b61-a240-23f79aeb5e78 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:51.380745 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1326/2641] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:27:51 2022] PUT /identity/v3/projects/33a1b243fddc47d386c9f6b5679af82b/users/ac07dedb10734d07921294f55cbf8be8/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:27:51.385193 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb55b9f-30ca-4b3a-9278-1283ecfc53f6 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:51.385665 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb55b9f-30ca-4b3a-9278-1283ecfc53f6 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:51.385665 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb55b9f-30ca-4b3a-9278-1283ecfc53f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:51.417220 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-3eb55b9f-30ca-4b3a-9278-1283ecfc53f6 None None] MFA Rules not processed for user `ac07dedb10734d07921294f55cbf8be8`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:27:51.418452 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-3eb55b9f-30ca-4b3a-9278-1283ecfc53f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:51.488052 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1316/2642] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:27:51 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:27:51.492868 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea29c124-b15c-4b61-a240-23f79aeb5e78 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:51.534032 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:27:51.563917 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea29c124-b15c-4b61-a240-23f79aeb5e78 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:51.569319 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d4db99-738e-4037-a062-410660a3cad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:51.570014 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d4db99-738e-4037-a062-410660a3cad0 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:51.570363 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d4db99-738e-4037-a062-410660a3cad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:51.570709 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d4db99-738e-4037-a062-410660a3cad0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:51.571438 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d4db99-738e-4037-a062-410660a3cad0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:51.573494 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d4db99-738e-4037-a062-410660a3cad0 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:51.604448 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1327/2643] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:27:51 2022] POST /identity/v3/projects => generated 403 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:27:51.608854 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb55b9f-30ca-4b3a-9278-1283ecfc53f6 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:51.621787 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb55b9f-30ca-4b3a-9278-1283ecfc53f6 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:51.670297 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:27:51.683031 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14ab5aa-7990-4235-8eff-95a3d4509b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:51.683599 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14ab5aa-7990-4235-8eff-95a3d4509b49 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:51.683873 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14ab5aa-7990-4235-8eff-95a3d4509b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:51.684139 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14ab5aa-7990-4235-8eff-95a3d4509b49 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:51.684776 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14ab5aa-7990-4235-8eff-95a3d4509b49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:51.686759 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14ab5aa-7990-4235-8eff-95a3d4509b49 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:51.711492 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1317/2644] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:27:51 2022] POST /identity/v3/users => generated 308 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:27:51.716540 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d4db99-738e-4037-a062-410660a3cad0 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:51.729271 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d4db99-738e-4037-a062-410660a3cad0 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:51.735143 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b2555a-f67b-429a-9072-3e67a7c6a473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:51.735825 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b2555a-f67b-429a-9072-3e67a7c6a473 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:51.736117 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b2555a-f67b-429a-9072-3e67a7c6a473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:51.736430 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b2555a-f67b-429a-9072-3e67a7c6a473 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:51.737149 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b2555a-f67b-429a-9072-3e67a7c6a473 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:51.739020 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b2555a-f67b-429a-9072-3e67a7c6a473 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:51.752059 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1328/2645] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:27:51 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:27:51.757948 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14ab5aa-7990-4235-8eff-95a3d4509b49 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:51.771929 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14ab5aa-7990-4235-8eff-95a3d4509b49 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:51.778097 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e65952d-9a7e-4481-a32a-00d4136fb78e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:51.779042 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e65952d-9a7e-4481-a32a-00d4136fb78e admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:51.779374 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e65952d-9a7e-4481-a32a-00d4136fb78e admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:51.779631 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e65952d-9a7e-4481-a32a-00d4136fb78e admin admin] PATH_INFO: `/v3/projects/beada951633b4fc5a9250fb002e10f7b/users/6957b4a65b5d48e8bf6b65dcdf855b99/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:51.796008 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e65952d-9a7e-4481-a32a-00d4136fb78e admin admin] RBAC: Authorizing `identity:create_grant(project_id=beada951633b4fc5a9250fb002e10f7b, user_id=6957b4a65b5d48e8bf6b65dcdf855b99, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:51.798700 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e65952d-9a7e-4481-a32a-00d4136fb78e admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:51.814048 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1318/2646] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:27:51 2022] PUT /identity/v3/projects/beada951633b4fc5a9250fb002e10f7b/users/6957b4a65b5d48e8bf6b65dcdf855b99/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:27:51.818977 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b2555a-f67b-429a-9072-3e67a7c6a473 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:51.829519 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b2555a-f67b-429a-9072-3e67a7c6a473 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:51.902175 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ad8fdf-77eb-4c4f-89d6-f4665bb0cd88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:51.902812 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad8fdf-77eb-4c4f-89d6-f4665bb0cd88 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:51.903100 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad8fdf-77eb-4c4f-89d6-f4665bb0cd88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:51.903471 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad8fdf-77eb-4c4f-89d6-f4665bb0cd88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:51.904174 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ad8fdf-77eb-4c4f-89d6-f4665bb0cd88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:51.905896 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ad8fdf-77eb-4c4f-89d6-f4665bb0cd88 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:51.916947 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1329/2647] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:27:51 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 16:27:51.921729 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e65952d-9a7e-4481-a32a-00d4136fb78e admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:51.933366 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e65952d-9a7e-4481-a32a-00d4136fb78e admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:51.938502 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7f7305-6da2-4934-bea1-1e88b6c03ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:51.939180 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f7305-6da2-4934-bea1-1e88b6c03ff3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:51.939525 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f7305-6da2-4934-bea1-1e88b6c03ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:51.939770 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f7305-6da2-4934-bea1-1e88b6c03ff3 admin admin] PATH_INFO: `/v3/projects/beada951633b4fc5a9250fb002e10f7b/users/6957b4a65b5d48e8bf6b65dcdf855b99/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:51.942124 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7f7305-6da2-4934-bea1-1e88b6c03ff3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=beada951633b4fc5a9250fb002e10f7b, user_id=6957b4a65b5d48e8bf6b65dcdf855b99, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:51.945010 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7f7305-6da2-4934-bea1-1e88b6c03ff3 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:51.956365 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1319/2648] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:27:51 2022] PUT /identity/v3/projects/beada951633b4fc5a9250fb002e10f7b/users/6957b4a65b5d48e8bf6b65dcdf855b99/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:27:51.967086 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc3cc95-ef0e-44d8-bd59-ab301f7d0ce7 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:51.967440 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc3cc95-ef0e-44d8-bd59-ab301f7d0ce7 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:51.967831 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc3cc95-ef0e-44d8-bd59-ab301f7d0ce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:52.004453 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-6fc3cc95-ef0e-44d8-bd59-ab301f7d0ce7 None None] MFA Rules not processed for user `6957b4a65b5d48e8bf6b65dcdf855b99`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:27:52.006161 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6fc3cc95-ef0e-44d8-bd59-ab301f7d0ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:52.090578 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1330/2649] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:27:51 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:27:52.102718 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7f7305-6da2-4934-bea1-1e88b6c03ff3 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:52.104717 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7c7f7305-6da2-4934-bea1-1e88b6c03ff3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:52.173148 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7f7305-6da2-4934-bea1-1e88b6c03ff3 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:52.177686 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60800bc4-c180-4763-b6e3-a86a2a3003c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:52.178295 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60800bc4-c180-4763-b6e3-a86a2a3003c9 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:52.178560 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60800bc4-c180-4763-b6e3-a86a2a3003c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:52.178814 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60800bc4-c180-4763-b6e3-a86a2a3003c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:52.189265 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60800bc4-c180-4763-b6e3-a86a2a3003c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:52.190718 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60800bc4-c180-4763-b6e3-a86a2a3003c9 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:52.200350 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1320/2650] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:27:52 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:27:52.367956 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc3cc95-ef0e-44d8-bd59-ab301f7d0ce7 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:52.421248 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:27:52.453554 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc3cc95-ef0e-44d8-bd59-ab301f7d0ce7 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:52.458346 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a037623e-4d95-4d05-b04f-33edf017fb4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:52.458987 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a037623e-4d95-4d05-b04f-33edf017fb4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:52.459262 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a037623e-4d95-4d05-b04f-33edf017fb4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:52.459531 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a037623e-4d95-4d05-b04f-33edf017fb4e admin admin] PATH_INFO: `/v3/users/ac07dedb10734d07921294f55cbf8be8` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:52.460664 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a037623e-4d95-4d05-b04f-33edf017fb4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac07dedb10734d07921294f55cbf8be8)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:52.462654 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a037623e-4d95-4d05-b04f-33edf017fb4e admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:52.540982 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a037623e-4d95-4d05-b04f-33edf017fb4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac07dedb10734d07921294f55cbf8be8'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:52.540982 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a037623e-4d95-4d05-b04f-33edf017fb4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac07dedb10734d07921294f55cbf8be8'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:52.540982 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a037623e-4d95-4d05-b04f-33edf017fb4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac07dedb10734d07921294f55cbf8be8'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:52.554255 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a037623e-4d95-4d05-b04f-33edf017fb4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac07dedb10734d07921294f55cbf8be8'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:52.555087 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a037623e-4d95-4d05-b04f-33edf017fb4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac07dedb10734d07921294f55cbf8be8'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:52.571541 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1331/2651] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:27:52 2022] DELETE /identity/v3/users/ac07dedb10734d07921294f55cbf8be8 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:27:52.576052 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c5876-89f9-4685-b9dd-62eb7bed52c5 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:52.576630 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c5876-89f9-4685-b9dd-62eb7bed52c5 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:52.576783 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c5876-89f9-4685-b9dd-62eb7bed52c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:52.600055 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-1e2c5876-89f9-4685-b9dd-62eb7bed52c5 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:27:52.601312 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-1e2c5876-89f9-4685-b9dd-62eb7bed52c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:52.668517 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1321/2652] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:27:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:27:52.678930 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a037623e-4d95-4d05-b04f-33edf017fb4e admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:52.680443 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a037623e-4d95-4d05-b04f-33edf017fb4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:52.744676 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a037623e-4d95-4d05-b04f-33edf017fb4e admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:52.752366 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e322e65c-8136-406d-8229-779eff09d609 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:52.752999 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e322e65c-8136-406d-8229-779eff09d609 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:52.753302 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e322e65c-8136-406d-8229-779eff09d609 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:52.753551 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e322e65c-8136-406d-8229-779eff09d609 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:52.763372 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e322e65c-8136-406d-8229-779eff09d609 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:52.764836 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e322e65c-8136-406d-8229-779eff09d609 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:52.774658 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1332/2653] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:27:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:27:53.061574 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2c5876-89f9-4685-b9dd-62eb7bed52c5 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:53.063711 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-1e2c5876-89f9-4685-b9dd-62eb7bed52c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:53.077673 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2c5876-89f9-4685-b9dd-62eb7bed52c5 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:53.083406 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c4af73-0192-4f77-b764-a181a90d036f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:53.084061 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4af73-0192-4f77-b764-a181a90d036f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:53.084333 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4af73-0192-4f77-b764-a181a90d036f admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:53.084600 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4af73-0192-4f77-b764-a181a90d036f admin admin] PATH_INFO: `/v3/projects/33a1b243fddc47d386c9f6b5679af82b` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:53.085783 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c4af73-0192-4f77-b764-a181a90d036f admin admin] RBAC: Authorizing `identity:delete_project(project_id=33a1b243fddc47d386c9f6b5679af82b)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:53.087720 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c4af73-0192-4f77-b764-a181a90d036f admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:53.135251 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:27:53.139984 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-f8c4af73-0192-4f77-b764-a181a90d036f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33a1b243fddc47d386c9f6b5679af82b', 'request_id': 'req-f8c4af73-0192-4f77-b764-a181a90d036f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:53.148973 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-f8c4af73-0192-4f77-b764-a181a90d036f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33a1b243fddc47d386c9f6b5679af82b', 'request_id': 'req-f8c4af73-0192-4f77-b764-a181a90d036f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:53.155191 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-f8c4af73-0192-4f77-b764-a181a90d036f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33a1b243fddc47d386c9f6b5679af82b', 'request_id': 'req-f8c4af73-0192-4f77-b764-a181a90d036f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:53.167194 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-f8c4af73-0192-4f77-b764-a181a90d036f admin admin] The token cache is being invalidate because project 33a1b243fddc47d386c9f6b5679af82b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:27:53.167477 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-f8c4af73-0192-4f77-b764-a181a90d036f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:53.168105 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-f8c4af73-0192-4f77-b764-a181a90d036f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:53.169848 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1322/2654] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:27:53 2022] DELETE /identity/v3/projects/33a1b243fddc47d386c9f6b5679af82b => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:27:53.175714 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e322e65c-8136-406d-8229-779eff09d609 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:53.187585 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e322e65c-8136-406d-8229-779eff09d609 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:53.237803 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:27:53.251068 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867a24b2-c7d9-421f-ad8e-649b82260c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:53.252455 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a24b2-c7d9-421f-ad8e-649b82260c2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:53.253059 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a24b2-c7d9-421f-ad8e-649b82260c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:53.254830 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a24b2-c7d9-421f-ad8e-649b82260c2c admin admin] PATH_INFO: `/v3/users/6957b4a65b5d48e8bf6b65dcdf855b99` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:53.256258 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867a24b2-c7d9-421f-ad8e-649b82260c2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6957b4a65b5d48e8bf6b65dcdf855b99)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:53.258453 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867a24b2-c7d9-421f-ad8e-649b82260c2c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:53.311167 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-867a24b2-c7d9-421f-ad8e-649b82260c2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6957b4a65b5d48e8bf6b65dcdf855b99'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:53.311823 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-867a24b2-c7d9-421f-ad8e-649b82260c2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6957b4a65b5d48e8bf6b65dcdf855b99'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:53.319353 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-867a24b2-c7d9-421f-ad8e-649b82260c2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6957b4a65b5d48e8bf6b65dcdf855b99'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:53.333215 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-867a24b2-c7d9-421f-ad8e-649b82260c2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6957b4a65b5d48e8bf6b65dcdf855b99'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:53.334096 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-867a24b2-c7d9-421f-ad8e-649b82260c2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6957b4a65b5d48e8bf6b65dcdf855b99'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:53.348985 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1333/2655] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:27:53 2022] DELETE /identity/v3/users/6957b4a65b5d48e8bf6b65dcdf855b99 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:27:53.661623 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c4af73-0192-4f77-b764-a181a90d036f admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:53.663526 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f8c4af73-0192-4f77-b764-a181a90d036f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:53.692365 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c4af73-0192-4f77-b764-a181a90d036f admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:53.769754 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ac5ad9-d5a5-49a5-becc-9c809e420fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:53.770942 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ac5ad9-d5a5-49a5-becc-9c809e420fa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:53.770942 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ac5ad9-d5a5-49a5-becc-9c809e420fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:53.770942 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ac5ad9-d5a5-49a5-becc-9c809e420fa1 admin admin] PATH_INFO: `/v3/projects/beada951633b4fc5a9250fb002e10f7b` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:53.772105 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ac5ad9-d5a5-49a5-becc-9c809e420fa1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=beada951633b4fc5a9250fb002e10f7b)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:53.774147 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ac5ad9-d5a5-49a5-becc-9c809e420fa1 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:53.827881 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-48ac5ad9-d5a5-49a5-becc-9c809e420fa1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'beada951633b4fc5a9250fb002e10f7b', 'request_id': 'req-48ac5ad9-d5a5-49a5-becc-9c809e420fa1'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:53.836348 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-48ac5ad9-d5a5-49a5-becc-9c809e420fa1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'beada951633b4fc5a9250fb002e10f7b', 'request_id': 'req-48ac5ad9-d5a5-49a5-becc-9c809e420fa1'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:53.842401 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-48ac5ad9-d5a5-49a5-becc-9c809e420fa1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'beada951633b4fc5a9250fb002e10f7b', 'request_id': 'req-48ac5ad9-d5a5-49a5-becc-9c809e420fa1'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:53.854305 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-48ac5ad9-d5a5-49a5-becc-9c809e420fa1 admin admin] The token cache is being invalidate because project beada951633b4fc5a9250fb002e10f7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:27:53.854587 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-48ac5ad9-d5a5-49a5-becc-9c809e420fa1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:53.855234 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-48ac5ad9-d5a5-49a5-becc-9c809e420fa1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:53.856718 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1323/2656] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:27:53 2022] DELETE /identity/v3/projects/beada951633b4fc5a9250fb002e10f7b => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:27:53.861761 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e756ce-24ae-4c65-bf5f-5af2bfc5a2d0 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:53.862234 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e756ce-24ae-4c65-bf5f-5af2bfc5a2d0 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:53.862759 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e756ce-24ae-4c65-bf5f-5af2bfc5a2d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:53.883647 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-b4e756ce-24ae-4c65-bf5f-5af2bfc5a2d0 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:27:53.885024 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b4e756ce-24ae-4c65-bf5f-5af2bfc5a2d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:53.945763 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1334/2657] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:27:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:27:53.954777 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9fdc0-c652-4544-8c6f-5c07f4bc6734 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:53.955099 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9fdc0-c652-4544-8c6f-5c07f4bc6734 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:53.955371 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9fdc0-c652-4544-8c6f-5c07f4bc6734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:53.975180 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-eed9fdc0-c652-4544-8c6f-5c07f4bc6734 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:27:53.976368 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-eed9fdc0-c652-4544-8c6f-5c07f4bc6734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:53.984424 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1324/2658] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:27:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:27:53.989085 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e756ce-24ae-4c65-bf5f-5af2bfc5a2d0 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:54.001075 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e756ce-24ae-4c65-bf5f-5af2bfc5a2d0 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:54.009543 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9810e963-53ca-46cc-bd96-a467a59cfc3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:54.010556 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9810e963-53ca-46cc-bd96-a467a59cfc3f admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:54.011048 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9810e963-53ca-46cc-bd96-a467a59cfc3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:54.011465 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9810e963-53ca-46cc-bd96-a467a59cfc3f admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:54.012578 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9810e963-53ca-46cc-bd96-a467a59cfc3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:27:54.013148 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9810e963-53ca-46cc-bd96-a467a59cfc3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:54.015824 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9810e963-53ca-46cc-bd96-a467a59cfc3f admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:54.028957 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1335/2659] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 6 16:27:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:27:54.033182 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed9fdc0-c652-4544-8c6f-5c07f4bc6734 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:54.042312 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed9fdc0-c652-4544-8c6f-5c07f4bc6734 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:54.047169 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8add10f0-508e-43bc-b755-e449d9efb7be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:54.047753 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add10f0-508e-43bc-b755-e449d9efb7be admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:54.048046 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add10f0-508e-43bc-b755-e449d9efb7be admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:54.048287 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add10f0-508e-43bc-b755-e449d9efb7be admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:54.049006 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8add10f0-508e-43bc-b755-e449d9efb7be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:54.051339 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8add10f0-508e-43bc-b755-e449d9efb7be admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:54.079973 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1325/2660] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:27:54 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:27:54.084858 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9810e963-53ca-46cc-bd96-a467a59cfc3f admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:54.100689 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9810e963-53ca-46cc-bd96-a467a59cfc3f admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:54.171311 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66b6f88-9429-4856-9bc1-5de85bc8149c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:54.172293 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66b6f88-9429-4856-9bc1-5de85bc8149c admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:54.172683 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66b6f88-9429-4856-9bc1-5de85bc8149c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:54.173124 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66b6f88-9429-4856-9bc1-5de85bc8149c admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:54.174049 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66b6f88-9429-4856-9bc1-5de85bc8149c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:54.176307 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66b6f88-9429-4856-9bc1-5de85bc8149c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:54.184466 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:27:54.201861 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1336/2661] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:27:54 2022] POST /identity/v3/users => generated 286 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:27:54.206258 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8add10f0-508e-43bc-b755-e449d9efb7be admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:54.217443 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8add10f0-508e-43bc-b755-e449d9efb7be admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:54.222832 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cabc0c-960e-4ec6-8f9e-97598380cfa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:54.223427 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cabc0c-960e-4ec6-8f9e-97598380cfa6 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:54.223720 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cabc0c-960e-4ec6-8f9e-97598380cfa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:54.223981 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cabc0c-960e-4ec6-8f9e-97598380cfa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:54.224651 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cabc0c-960e-4ec6-8f9e-97598380cfa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:54.226314 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cabc0c-960e-4ec6-8f9e-97598380cfa6 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:54.236926 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1326/2662] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:27:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:27:54.241676 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66b6f88-9429-4856-9bc1-5de85bc8149c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:54.253432 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66b6f88-9429-4856-9bc1-5de85bc8149c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:54.259667 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d806a7c4-7817-46f7-820a-71ffc1db28c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:54.260576 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d806a7c4-7817-46f7-820a-71ffc1db28c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:54.260922 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d806a7c4-7817-46f7-820a-71ffc1db28c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:54.261223 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d806a7c4-7817-46f7-820a-71ffc1db28c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:54.262161 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d806a7c4-7817-46f7-820a-71ffc1db28c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:54.264495 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d806a7c4-7817-46f7-820a-71ffc1db28c3 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:54.279577 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1337/2663] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:27:54 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:27:54.284299 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cabc0c-960e-4ec6-8f9e-97598380cfa6 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:54.295399 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cabc0c-960e-4ec6-8f9e-97598380cfa6 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:54.301308 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81facacc-8fcf-4245-bf1d-634a6f7035ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:54.301969 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81facacc-8fcf-4245-bf1d-634a6f7035ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:54.302245 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81facacc-8fcf-4245-bf1d-634a6f7035ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:54.302499 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81facacc-8fcf-4245-bf1d-634a6f7035ae admin admin] PATH_INFO: `/v3/projects/399c6b5901ca40bf8a158f8f7f4144f0/users/17168974cb4548cfb26c62b703e9757b/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:54.318113 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81facacc-8fcf-4245-bf1d-634a6f7035ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=399c6b5901ca40bf8a158f8f7f4144f0, user_id=17168974cb4548cfb26c62b703e9757b, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:54.320790 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81facacc-8fcf-4245-bf1d-634a6f7035ae admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:54.329855 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:27:54.337789 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1327/2664] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:27:54 2022] PUT /identity/v3/projects/399c6b5901ca40bf8a158f8f7f4144f0/users/17168974cb4548cfb26c62b703e9757b/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:27:54.349014 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee8b53-0ab4-42e9-90f3-c2b772bf610c None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:54.349293 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee8b53-0ab4-42e9-90f3-c2b772bf610c None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:54.349560 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee8b53-0ab4-42e9-90f3-c2b772bf610c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:54.378399 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-6eee8b53-0ab4-42e9-90f3-c2b772bf610c None None] MFA Rules not processed for user `17168974cb4548cfb26c62b703e9757b`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:27:54.379631 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6eee8b53-0ab4-42e9-90f3-c2b772bf610c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:54.452190 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1338/2665] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:27:54 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:27:54.457081 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81facacc-8fcf-4245-bf1d-634a6f7035ae admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:54.535228 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81facacc-8fcf-4245-bf1d-634a6f7035ae admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:54.540300 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db86f0cb-d674-4a5d-909c-3ecd2dcf9037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:54.540962 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db86f0cb-d674-4a5d-909c-3ecd2dcf9037 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:54.541235 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db86f0cb-d674-4a5d-909c-3ecd2dcf9037 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:54.541503 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db86f0cb-d674-4a5d-909c-3ecd2dcf9037 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:54.542141 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db86f0cb-d674-4a5d-909c-3ecd2dcf9037 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:54.544036 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db86f0cb-d674-4a5d-909c-3ecd2dcf9037 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:54.573403 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1328/2666] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:27:54 2022] POST /identity/v3/projects => generated 371 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:27:54.577956 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eee8b53-0ab4-42e9-90f3-c2b772bf610c None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:54.589118 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eee8b53-0ab4-42e9-90f3-c2b772bf610c None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:54.652967 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea9bd73-c5f6-4afd-9b0e-983489fb37d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:54.653608 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea9bd73-c5f6-4afd-9b0e-983489fb37d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:54.653907 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea9bd73-c5f6-4afd-9b0e-983489fb37d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:54.654164 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea9bd73-c5f6-4afd-9b0e-983489fb37d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:54.654842 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea9bd73-c5f6-4afd-9b0e-983489fb37d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:54.656855 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea9bd73-c5f6-4afd-9b0e-983489fb37d9 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:54.682435 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1339/2667] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:27:54 2022] POST /identity/v3/users => generated 292 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:27:54.694636 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db86f0cb-d674-4a5d-909c-3ecd2dcf9037 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:54.707121 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db86f0cb-d674-4a5d-909c-3ecd2dcf9037 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:54.716846 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56074d4b-d14a-4905-8d0e-a9b09f98e2e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:54.716846 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56074d4b-d14a-4905-8d0e-a9b09f98e2e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:54.716846 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56074d4b-d14a-4905-8d0e-a9b09f98e2e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:54.716846 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56074d4b-d14a-4905-8d0e-a9b09f98e2e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:54.717315 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56074d4b-d14a-4905-8d0e-a9b09f98e2e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:54.718893 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56074d4b-d14a-4905-8d0e-a9b09f98e2e0 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:54.725001 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea9bd73-c5f6-4afd-9b0e-983489fb37d9 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:54.728712 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-7ea9bd73-c5f6-4afd-9b0e-983489fb37d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:54.735627 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1329/2668] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:27:54 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:27:54.737918 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56074d4b-d14a-4905-8d0e-a9b09f98e2e0 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:54.754661 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56074d4b-d14a-4905-8d0e-a9b09f98e2e0 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:54.760438 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a68f400-0185-414d-8254-9c9484530ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:54.763260 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a68f400-0185-414d-8254-9c9484530ec0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:54.763260 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a68f400-0185-414d-8254-9c9484530ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:54.763260 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a68f400-0185-414d-8254-9c9484530ec0 admin admin] PATH_INFO: `/v3/projects/d652f67178ae44a3875eadabd792cde3/users/3e62ee38ca5b42988e4d9a2c65f657b2/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:54.780616 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a68f400-0185-414d-8254-9c9484530ec0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d652f67178ae44a3875eadabd792cde3, user_id=3e62ee38ca5b42988e4d9a2c65f657b2, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:54.783998 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a68f400-0185-414d-8254-9c9484530ec0 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:54.792829 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:27:54.808098 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1330/2669] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:27:54 2022] PUT /identity/v3/projects/d652f67178ae44a3875eadabd792cde3/users/3e62ee38ca5b42988e4d9a2c65f657b2/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:27:54.813253 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a68f400-0185-414d-8254-9c9484530ec0 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:54.827027 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea9bd73-c5f6-4afd-9b0e-983489fb37d9 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:54.917149 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1fcbfa-f7c5-4901-b3c8-9c2ad5da22a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63de41f4348243998029f7d07aa5dc40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:54.917919 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1fcbfa-f7c5-4901-b3c8-9c2ad5da22a9 service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:54.918143 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1fcbfa-f7c5-4901-b3c8-9c2ad5da22a9 service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:54.918535 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1fcbfa-f7c5-4901-b3c8-9c2ad5da22a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:54.920684 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6d1fcbfa-f7c5-4901-b3c8-9c2ad5da22a9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:54.932870 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a68f400-0185-414d-8254-9c9484530ec0 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:54.934153 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1fcbfa-f7c5-4901-b3c8-9c2ad5da22a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:54.935989 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1fcbfa-f7c5-4901-b3c8-9c2ad5da22a9 service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:54.940073 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9fb1bd-a91c-40f5-bb9e-540a9d19f055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:54.943262 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9fb1bd-a91c-40f5-bb9e-540a9d19f055 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:54.943759 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9fb1bd-a91c-40f5-bb9e-540a9d19f055 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:54.944116 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9fb1bd-a91c-40f5-bb9e-540a9d19f055 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:54.945298 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9fb1bd-a91c-40f5-bb9e-540a9d19f055 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:54.947876 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9fb1bd-a91c-40f5-bb9e-540a9d19f055 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:54.949623 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1340/2670] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:27:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:27:54.970625 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1331/2671] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:27:54 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:27:54.975970 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1fcbfa-f7c5-4901-b3c8-9c2ad5da22a9 service nova] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:54.988927 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1fcbfa-f7c5-4901-b3c8-9c2ad5da22a9 service nova] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:54.994430 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8615f6c9-82b6-4dff-8fa9-4fb766ea0ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:54.995134 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8615f6c9-82b6-4dff-8fa9-4fb766ea0ed0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:54.995422 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8615f6c9-82b6-4dff-8fa9-4fb766ea0ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:54.995695 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8615f6c9-82b6-4dff-8fa9-4fb766ea0ed0 admin admin] PATH_INFO: `/v3/projects/d652f67178ae44a3875eadabd792cde3/users/3e62ee38ca5b42988e4d9a2c65f657b2/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:54.997785 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8615f6c9-82b6-4dff-8fa9-4fb766ea0ed0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d652f67178ae44a3875eadabd792cde3, user_id=3e62ee38ca5b42988e4d9a2c65f657b2, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:55.000677 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8615f6c9-82b6-4dff-8fa9-4fb766ea0ed0 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:55.012293 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1341/2672] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:27:54 2022] PUT /identity/v3/projects/d652f67178ae44a3875eadabd792cde3/users/3e62ee38ca5b42988e4d9a2c65f657b2/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:27:55.023962 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae900ba-202b-4350-98af-d2929afb0e11 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:55.024243 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae900ba-202b-4350-98af-d2929afb0e11 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:55.024513 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae900ba-202b-4350-98af-d2929afb0e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:55.055945 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-eae900ba-202b-4350-98af-d2929afb0e11 None None] MFA Rules not processed for user `3e62ee38ca5b42988e4d9a2c65f657b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:27:55.057691 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-eae900ba-202b-4350-98af-d2929afb0e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:55.094468 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:27:55.158099 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1332/2673] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:27:55 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:27:55.181459 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8615f6c9-82b6-4dff-8fa9-4fb766ea0ed0 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:55.184626 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-8615f6c9-82b6-4dff-8fa9-4fb766ea0ed0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:55.269174 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8615f6c9-82b6-4dff-8fa9-4fb766ea0ed0 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:55.273363 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa0f159-5bb5-4ba4-9dda-45e6f81ec63e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:55.274516 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa0f159-5bb5-4ba4-9dda-45e6f81ec63e service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:55.274516 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa0f159-5bb5-4ba4-9dda-45e6f81ec63e service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:55.274516 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa0f159-5bb5-4ba4-9dda-45e6f81ec63e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:55.288853 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa0f159-5bb5-4ba4-9dda-45e6f81ec63e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:55.290541 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa0f159-5bb5-4ba4-9dda-45e6f81ec63e service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:55.306301 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1342/2674] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:27:55 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:27:55.756591 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae900ba-202b-4350-98af-d2929afb0e11 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:55.825505 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae900ba-202b-4350-98af-d2929afb0e11 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:55.831199 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f3f115-e7c5-4589-b961-654af17054bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:55.831838 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f3f115-e7c5-4589-b961-654af17054bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:55.832113 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f3f115-e7c5-4589-b961-654af17054bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:55.832382 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f3f115-e7c5-4589-b961-654af17054bf admin admin] PATH_INFO: `/v3/users/17168974cb4548cfb26c62b703e9757b` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:55.833870 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f3f115-e7c5-4589-b961-654af17054bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=17168974cb4548cfb26c62b703e9757b)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:55.835830 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f3f115-e7c5-4589-b961-654af17054bf admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:55.889323 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-54f3f115-e7c5-4589-b961-654af17054bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17168974cb4548cfb26c62b703e9757b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:55.889993 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-54f3f115-e7c5-4589-b961-654af17054bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17168974cb4548cfb26c62b703e9757b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:55.898030 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-54f3f115-e7c5-4589-b961-654af17054bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17168974cb4548cfb26c62b703e9757b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:55.911069 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-54f3f115-e7c5-4589-b961-654af17054bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17168974cb4548cfb26c62b703e9757b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:55.921155 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-54f3f115-e7c5-4589-b961-654af17054bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17168974cb4548cfb26c62b703e9757b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:55.928075 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1333/2675] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:27:55 2022] DELETE /identity/v3/users/17168974cb4548cfb26c62b703e9757b => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:27:55.932783 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9c2d40-c305-4ec5-b7ed-8384d098f80c None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:55.933268 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9c2d40-c305-4ec5-b7ed-8384d098f80c None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:55.933387 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9c2d40-c305-4ec5-b7ed-8384d098f80c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:55.953407 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-0f9c2d40-c305-4ec5-b7ed-8384d098f80c None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:27:55.954705 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0f9c2d40-c305-4ec5-b7ed-8384d098f80c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:56.021253 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1343/2676] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:27:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:27:56.035032 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f3f115-e7c5-4589-b961-654af17054bf admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:56.036716 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-54f3f115-e7c5-4589-b961-654af17054bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:56.102307 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:27:56.106515 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f3f115-e7c5-4589-b961-654af17054bf admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:56.111127 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb79c31f-d97a-461c-9c64-15bf972ad670 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:56.112120 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb79c31f-d97a-461c-9c64-15bf972ad670 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:56.112641 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb79c31f-d97a-461c-9c64-15bf972ad670 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:56.113253 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb79c31f-d97a-461c-9c64-15bf972ad670 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:56.123689 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb79c31f-d97a-461c-9c64-15bf972ad670 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:56.125241 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb79c31f-d97a-461c-9c64-15bf972ad670 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:56.136687 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1334/2677] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:27:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:27:56.507820 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9c2d40-c305-4ec5-b7ed-8384d098f80c None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:56.509802 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0f9c2d40-c305-4ec5-b7ed-8384d098f80c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:56.529438 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9c2d40-c305-4ec5-b7ed-8384d098f80c None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:56.534605 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432b486a-9748-4aac-9995-669c108ca149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:56.535292 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432b486a-9748-4aac-9995-669c108ca149 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:56.535960 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432b486a-9748-4aac-9995-669c108ca149 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:56.535960 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432b486a-9748-4aac-9995-669c108ca149 admin admin] PATH_INFO: `/v3/projects/399c6b5901ca40bf8a158f8f7f4144f0` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:56.536996 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432b486a-9748-4aac-9995-669c108ca149 admin admin] RBAC: Authorizing `identity:delete_project(project_id=399c6b5901ca40bf8a158f8f7f4144f0)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:56.538984 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432b486a-9748-4aac-9995-669c108ca149 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:56.547211 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:27:56.602682 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-432b486a-9748-4aac-9995-669c108ca149 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '399c6b5901ca40bf8a158f8f7f4144f0', 'request_id': 'req-432b486a-9748-4aac-9995-669c108ca149'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:56.609028 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-432b486a-9748-4aac-9995-669c108ca149 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '399c6b5901ca40bf8a158f8f7f4144f0', 'request_id': 'req-432b486a-9748-4aac-9995-669c108ca149'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:56.618305 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-432b486a-9748-4aac-9995-669c108ca149 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '399c6b5901ca40bf8a158f8f7f4144f0', 'request_id': 'req-432b486a-9748-4aac-9995-669c108ca149'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:56.631524 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-432b486a-9748-4aac-9995-669c108ca149 admin admin] The token cache is being invalidate because project 399c6b5901ca40bf8a158f8f7f4144f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:27:56.632189 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-432b486a-9748-4aac-9995-669c108ca149 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:56.633352 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-432b486a-9748-4aac-9995-669c108ca149 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:56.635210 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1344/2678] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:27:56 2022] DELETE /identity/v3/projects/399c6b5901ca40bf8a158f8f7f4144f0 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:27:56.639309 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb79c31f-d97a-461c-9c64-15bf972ad670 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:56.649546 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb79c31f-d97a-461c-9c64-15bf972ad670 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:56.713054 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da267f31-ad13-426b-af51-4648b6dc40e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:56.714276 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da267f31-ad13-426b-af51-4648b6dc40e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:56.714785 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da267f31-ad13-426b-af51-4648b6dc40e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:56.715412 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da267f31-ad13-426b-af51-4648b6dc40e8 admin admin] PATH_INFO: `/v3/users/3e62ee38ca5b42988e4d9a2c65f657b2` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:56.717086 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da267f31-ad13-426b-af51-4648b6dc40e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e62ee38ca5b42988e4d9a2c65f657b2)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:56.719116 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da267f31-ad13-426b-af51-4648b6dc40e8 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:56.784005 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-da267f31-ad13-426b-af51-4648b6dc40e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e62ee38ca5b42988e4d9a2c65f657b2'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:56.785904 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-da267f31-ad13-426b-af51-4648b6dc40e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e62ee38ca5b42988e4d9a2c65f657b2'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:56.793619 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-da267f31-ad13-426b-af51-4648b6dc40e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e62ee38ca5b42988e4d9a2c65f657b2'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:56.809191 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-da267f31-ad13-426b-af51-4648b6dc40e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e62ee38ca5b42988e4d9a2c65f657b2'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:56.817458 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-da267f31-ad13-426b-af51-4648b6dc40e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e62ee38ca5b42988e4d9a2c65f657b2'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:56.823591 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1335/2679] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:27:56 2022] DELETE /identity/v3/users/3e62ee38ca5b42988e4d9a2c65f657b2 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:27:57.127515 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432b486a-9748-4aac-9995-669c108ca149 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:57.129684 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-432b486a-9748-4aac-9995-669c108ca149 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:57.146113 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432b486a-9748-4aac-9995-669c108ca149 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:57.217775 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbf3bfb-b989-47f2-b998-155813528553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:57.218487 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf3bfb-b989-47f2-b998-155813528553 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:57.218800 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf3bfb-b989-47f2-b998-155813528553 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:57.219068 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf3bfb-b989-47f2-b998-155813528553 admin admin] PATH_INFO: `/v3/projects/d652f67178ae44a3875eadabd792cde3` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:57.220343 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbf3bfb-b989-47f2-b998-155813528553 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d652f67178ae44a3875eadabd792cde3)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:57.222480 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbf3bfb-b989-47f2-b998-155813528553 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:57.284137 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5cbf3bfb-b989-47f2-b998-155813528553 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd652f67178ae44a3875eadabd792cde3', 'request_id': 'req-5cbf3bfb-b989-47f2-b998-155813528553'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:57.290557 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5cbf3bfb-b989-47f2-b998-155813528553 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd652f67178ae44a3875eadabd792cde3', 'request_id': 'req-5cbf3bfb-b989-47f2-b998-155813528553'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:57.298374 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5cbf3bfb-b989-47f2-b998-155813528553 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd652f67178ae44a3875eadabd792cde3', 'request_id': 'req-5cbf3bfb-b989-47f2-b998-155813528553'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:57.312015 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5cbf3bfb-b989-47f2-b998-155813528553 admin admin] The token cache is being invalidate because project d652f67178ae44a3875eadabd792cde3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:27:57.312308 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5cbf3bfb-b989-47f2-b998-155813528553 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:57.313004 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5cbf3bfb-b989-47f2-b998-155813528553 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:57.314906 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1345/2680] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:27:57 2022] DELETE /identity/v3/projects/d652f67178ae44a3875eadabd792cde3 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:27:57.320978 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425ecab8-420c-4588-8f2d-68d2fd1476c8 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:57.321314 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425ecab8-420c-4588-8f2d-68d2fd1476c8 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:57.321549 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425ecab8-420c-4588-8f2d-68d2fd1476c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:57.342012 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-425ecab8-420c-4588-8f2d-68d2fd1476c8 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:27:57.343298 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-425ecab8-420c-4588-8f2d-68d2fd1476c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:57.413875 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1336/2681] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:27:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:27:57.422218 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c4861-abff-4d81-a76c-7de36d6116cc None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:57.422517 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c4861-abff-4d81-a76c-7de36d6116cc None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:57.422801 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c4861-abff-4d81-a76c-7de36d6116cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:57.446340 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-218c4861-abff-4d81-a76c-7de36d6116cc None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:27:57.447870 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-218c4861-abff-4d81-a76c-7de36d6116cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:57.456999 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1346/2682] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:27:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:27:57.462779 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425ecab8-420c-4588-8f2d-68d2fd1476c8 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:57.475202 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425ecab8-420c-4588-8f2d-68d2fd1476c8 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:57.480697 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a866cc-6c45-4c6a-a07a-2ad8a48365d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:57.481456 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a866cc-6c45-4c6a-a07a-2ad8a48365d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:57.481800 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a866cc-6c45-4c6a-a07a-2ad8a48365d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:57.482168 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a866cc-6c45-4c6a-a07a-2ad8a48365d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:57.482913 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a866cc-6c45-4c6a-a07a-2ad8a48365d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:27:57.483276 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a866cc-6c45-4c6a-a07a-2ad8a48365d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:57.485126 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a866cc-6c45-4c6a-a07a-2ad8a48365d2 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:57.493237 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:27:57.500034 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1337/2683] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 6 16:27:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:27:57.505134 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218c4861-abff-4d81-a76c-7de36d6116cc None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:57.516058 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218c4861-abff-4d81-a76c-7de36d6116cc None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:57.520940 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4327b87c-786f-4e8b-9abc-b1e5b18cabcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:57.521608 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4327b87c-786f-4e8b-9abc-b1e5b18cabcc admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:57.521883 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4327b87c-786f-4e8b-9abc-b1e5b18cabcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:57.522147 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4327b87c-786f-4e8b-9abc-b1e5b18cabcc admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:57.523154 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4327b87c-786f-4e8b-9abc-b1e5b18cabcc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:57.524821 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4327b87c-786f-4e8b-9abc-b1e5b18cabcc admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:57.543256 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:27:57.564536 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1347/2684] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:27:57 2022] POST /identity/v3/projects => generated 363 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:27:57.566512 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a866cc-6c45-4c6a-a07a-2ad8a48365d2 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:57.578201 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a866cc-6c45-4c6a-a07a-2ad8a48365d2 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:57.639720 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c678c959-1da0-4efb-9f55-a908cfb45226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:57.640332 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c678c959-1da0-4efb-9f55-a908cfb45226 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:57.640601 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c678c959-1da0-4efb-9f55-a908cfb45226 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:57.640905 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c678c959-1da0-4efb-9f55-a908cfb45226 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:57.641554 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c678c959-1da0-4efb-9f55-a908cfb45226 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:57.643470 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c678c959-1da0-4efb-9f55-a908cfb45226 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:57.668880 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1338/2685] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:27:57 2022] POST /identity/v3/users => generated 282 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:27:57.672974 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4327b87c-786f-4e8b-9abc-b1e5b18cabcc admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:57.683914 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4327b87c-786f-4e8b-9abc-b1e5b18cabcc admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:57.690390 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413761e8-5525-4bca-8b45-c5a281bdca93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:57.691304 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413761e8-5525-4bca-8b45-c5a281bdca93 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:57.691654 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413761e8-5525-4bca-8b45-c5a281bdca93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:57.692079 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413761e8-5525-4bca-8b45-c5a281bdca93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:57.693071 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413761e8-5525-4bca-8b45-c5a281bdca93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:57.695421 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413761e8-5525-4bca-8b45-c5a281bdca93 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:57.709911 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1348/2686] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:27:57 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 16:27:57.714421 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c678c959-1da0-4efb-9f55-a908cfb45226 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:57.727981 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c678c959-1da0-4efb-9f55-a908cfb45226 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:57.732698 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633bdcb4-1254-4824-81ff-22dbd5d4c5b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:57.733494 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633bdcb4-1254-4824-81ff-22dbd5d4c5b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:57.733767 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633bdcb4-1254-4824-81ff-22dbd5d4c5b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:57.734076 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633bdcb4-1254-4824-81ff-22dbd5d4c5b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:57.734678 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633bdcb4-1254-4824-81ff-22dbd5d4c5b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:57.736310 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633bdcb4-1254-4824-81ff-22dbd5d4c5b8 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:57.750258 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1339/2687] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:27:57 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:27:57.754785 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413761e8-5525-4bca-8b45-c5a281bdca93 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:57.765636 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413761e8-5525-4bca-8b45-c5a281bdca93 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:57.771418 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6ee860-1d31-42fb-a414-40ea0943c65f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:57.772156 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6ee860-1d31-42fb-a414-40ea0943c65f admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:57.772522 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6ee860-1d31-42fb-a414-40ea0943c65f admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:57.772895 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6ee860-1d31-42fb-a414-40ea0943c65f admin admin] PATH_INFO: `/v3/projects/e03d47527fff465b96b708eea9078cc7/users/b78cda00aecd4f57a1aab8852b393d6d/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:57.789501 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6ee860-1d31-42fb-a414-40ea0943c65f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e03d47527fff465b96b708eea9078cc7, user_id=b78cda00aecd4f57a1aab8852b393d6d, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:57.792277 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6ee860-1d31-42fb-a414-40ea0943c65f admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:57.806285 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1349/2688] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:27:57 2022] PUT /identity/v3/projects/e03d47527fff465b96b708eea9078cc7/users/b78cda00aecd4f57a1aab8852b393d6d/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:27:57.816876 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3987ffc9-96b7-4843-9097-ddbadaa64a44 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:57.817376 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3987ffc9-96b7-4843-9097-ddbadaa64a44 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:57.817476 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3987ffc9-96b7-4843-9097-ddbadaa64a44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:57.845717 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-3987ffc9-96b7-4843-9097-ddbadaa64a44 None None] MFA Rules not processed for user `b78cda00aecd4f57a1aab8852b393d6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:27:57.847054 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-3987ffc9-96b7-4843-9097-ddbadaa64a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:57.921843 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1340/2689] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:27:57 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:27:57.923560 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6ee860-1d31-42fb-a414-40ea0943c65f admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:57.986476 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6ee860-1d31-42fb-a414-40ea0943c65f admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:57.992219 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45376454-abc0-42f8-aacc-d57b6254d76e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:57.992863 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45376454-abc0-42f8-aacc-d57b6254d76e admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:57.993163 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45376454-abc0-42f8-aacc-d57b6254d76e admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:57.993436 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45376454-abc0-42f8-aacc-d57b6254d76e admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:57.994058 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45376454-abc0-42f8-aacc-d57b6254d76e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:57.995979 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45376454-abc0-42f8-aacc-d57b6254d76e admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:58.026297 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1350/2690] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:27:57 2022] POST /identity/v3/projects => generated 365 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:27:58.030602 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3987ffc9-96b7-4843-9097-ddbadaa64a44 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:58.045325 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3987ffc9-96b7-4843-9097-ddbadaa64a44 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:58.104654 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05eb1de-3465-40b3-af73-042ca6424633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:58.105419 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05eb1de-3465-40b3-af73-042ca6424633 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:58.105800 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05eb1de-3465-40b3-af73-042ca6424633 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:58.106076 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05eb1de-3465-40b3-af73-042ca6424633 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:58.106708 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05eb1de-3465-40b3-af73-042ca6424633 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:58.108665 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05eb1de-3465-40b3-af73-042ca6424633 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:58.121483 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:27:58.138490 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1341/2691] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:27:58 2022] POST /identity/v3/users => generated 289 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:27:58.142234 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45376454-abc0-42f8-aacc-d57b6254d76e admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:58.153039 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45376454-abc0-42f8-aacc-d57b6254d76e admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:58.158230 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24995f1-55f1-4c61-af18-812f5dc2d604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:58.158840 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24995f1-55f1-4c61-af18-812f5dc2d604 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:58.159173 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24995f1-55f1-4c61-af18-812f5dc2d604 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:58.159465 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24995f1-55f1-4c61-af18-812f5dc2d604 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:58.160220 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24995f1-55f1-4c61-af18-812f5dc2d604 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:58.161881 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24995f1-55f1-4c61-af18-812f5dc2d604 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:58.182030 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1351/2692] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:27:58 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:27:58.186516 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05eb1de-3465-40b3-af73-042ca6424633 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:58.203852 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05eb1de-3465-40b3-af73-042ca6424633 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:58.211659 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141c0ede-2545-4203-bc00-fc0af6191aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:58.212721 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c0ede-2545-4203-bc00-fc0af6191aea admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:58.213202 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c0ede-2545-4203-bc00-fc0af6191aea admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:58.213670 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c0ede-2545-4203-bc00-fc0af6191aea admin admin] PATH_INFO: `/v3/projects/8061b0883ff4404b96e9e2173577f3b7/users/f6389340be944a4da816567f178eb2b2/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:58.238644 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141c0ede-2545-4203-bc00-fc0af6191aea admin admin] RBAC: Authorizing `identity:create_grant(project_id=8061b0883ff4404b96e9e2173577f3b7, user_id=f6389340be944a4da816567f178eb2b2, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:58.243788 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141c0ede-2545-4203-bc00-fc0af6191aea admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:58.261887 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1342/2693] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:27:58 2022] PUT /identity/v3/projects/8061b0883ff4404b96e9e2173577f3b7/users/f6389340be944a4da816567f178eb2b2/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:27:58.266734 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24995f1-55f1-4c61-af18-812f5dc2d604 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:58.283697 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24995f1-55f1-4c61-af18-812f5dc2d604 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:58.350172 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005b95b6-afbe-42bd-acb2-304ca33f94b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:58.350881 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b95b6-afbe-42bd-acb2-304ca33f94b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:58.351226 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b95b6-afbe-42bd-acb2-304ca33f94b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:58.351563 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b95b6-afbe-42bd-acb2-304ca33f94b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:58.352329 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005b95b6-afbe-42bd-acb2-304ca33f94b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:58.354249 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005b95b6-afbe-42bd-acb2-304ca33f94b6 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:58.368713 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1352/2694] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:27:58 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:27:58.370039 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141c0ede-2545-4203-bc00-fc0af6191aea admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:58.380663 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141c0ede-2545-4203-bc00-fc0af6191aea admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:58.386590 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975faf13-8950-4579-93c2-1c585e6a077e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:58.387355 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975faf13-8950-4579-93c2-1c585e6a077e admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:58.387543 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975faf13-8950-4579-93c2-1c585e6a077e admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:58.387817 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975faf13-8950-4579-93c2-1c585e6a077e admin admin] PATH_INFO: `/v3/projects/8061b0883ff4404b96e9e2173577f3b7/users/f6389340be944a4da816567f178eb2b2/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:58.389810 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975faf13-8950-4579-93c2-1c585e6a077e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8061b0883ff4404b96e9e2173577f3b7, user_id=f6389340be944a4da816567f178eb2b2, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:58.392533 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975faf13-8950-4579-93c2-1c585e6a077e admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:58.404475 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1343/2695] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:27:58 2022] PUT /identity/v3/projects/8061b0883ff4404b96e9e2173577f3b7/users/f6389340be944a4da816567f178eb2b2/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:27:58.417158 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44bb7f0-6b69-4204-a452-14b7aa3abb62 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:58.417841 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44bb7f0-6b69-4204-a452-14b7aa3abb62 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:58.418326 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44bb7f0-6b69-4204-a452-14b7aa3abb62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:58.456636 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-f44bb7f0-6b69-4204-a452-14b7aa3abb62 None None] MFA Rules not processed for user `f6389340be944a4da816567f178eb2b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:27:58.457902 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f44bb7f0-6b69-4204-a452-14b7aa3abb62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:58.479380 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:27:58.539489 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1353/2696] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:27:58 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:27:58.551504 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975faf13-8950-4579-93c2-1c585e6a077e admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:58.553331 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-975faf13-8950-4579-93c2-1c585e6a077e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:58.614977 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975faf13-8950-4579-93c2-1c585e6a077e admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:58.618750 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e03f35a-930e-4d32-a14d-56dd2e952c49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:58.619621 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e03f35a-930e-4d32-a14d-56dd2e952c49 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:58.619621 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e03f35a-930e-4d32-a14d-56dd2e952c49 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:58.619902 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e03f35a-930e-4d32-a14d-56dd2e952c49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:58.629601 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e03f35a-930e-4d32-a14d-56dd2e952c49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:58.631143 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e03f35a-930e-4d32-a14d-56dd2e952c49 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:58.641370 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1344/2697] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:27:58 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:27:58.721432 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44bb7f0-6b69-4204-a452-14b7aa3abb62 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:58.732461 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44bb7f0-6b69-4204-a452-14b7aa3abb62 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:58.737033 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1872a891-e3b9-466d-93f7-2634ed0ba68a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:58.737811 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1872a891-e3b9-466d-93f7-2634ed0ba68a service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:58.738145 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1872a891-e3b9-466d-93f7-2634ed0ba68a service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:58.738476 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1872a891-e3b9-466d-93f7-2634ed0ba68a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:58.805684 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1872a891-e3b9-466d-93f7-2634ed0ba68a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:58.807341 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1872a891-e3b9-466d-93f7-2634ed0ba68a service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:58.818052 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1354/2698] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:27:58 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:28:08.637219 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=43,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:28:08.812584 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=48 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:28:10.364168 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e03f35a-930e-4d32-a14d-56dd2e952c49 service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:10.427020 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e03f35a-930e-4d32-a14d-56dd2e952c49 service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:10.431805 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121e4cdc-b2ce-4754-bff1-34bb9f9dcc7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:10.432633 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121e4cdc-b2ce-4754-bff1-34bb9f9dcc7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:10.433106 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121e4cdc-b2ce-4754-bff1-34bb9f9dcc7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:10.433494 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121e4cdc-b2ce-4754-bff1-34bb9f9dcc7a admin admin] PATH_INFO: `/v3/users/b78cda00aecd4f57a1aab8852b393d6d` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:10.434872 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121e4cdc-b2ce-4754-bff1-34bb9f9dcc7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b78cda00aecd4f57a1aab8852b393d6d)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:10.436930 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121e4cdc-b2ce-4754-bff1-34bb9f9dcc7a admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:10.492312 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-121e4cdc-b2ce-4754-bff1-34bb9f9dcc7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b78cda00aecd4f57a1aab8852b393d6d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:10.493005 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-121e4cdc-b2ce-4754-bff1-34bb9f9dcc7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b78cda00aecd4f57a1aab8852b393d6d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:10.500792 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-121e4cdc-b2ce-4754-bff1-34bb9f9dcc7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b78cda00aecd4f57a1aab8852b393d6d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:10.512712 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-121e4cdc-b2ce-4754-bff1-34bb9f9dcc7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b78cda00aecd4f57a1aab8852b393d6d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:10.523048 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-121e4cdc-b2ce-4754-bff1-34bb9f9dcc7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b78cda00aecd4f57a1aab8852b393d6d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:10.529977 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1345/2699] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:28:10 2022] DELETE /identity/v3/users/b78cda00aecd4f57a1aab8852b393d6d => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:28:10.535014 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b395d3-a1bb-402a-a5ca-79e9608bbf83 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:10.535655 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b395d3-a1bb-402a-a5ca-79e9608bbf83 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:10.535655 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b395d3-a1bb-402a-a5ca-79e9608bbf83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:10.557103 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-a5b395d3-a1bb-402a-a5ca-79e9608bbf83 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:28:10.558339 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a5b395d3-a1bb-402a-a5ca-79e9608bbf83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:10.626030 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1355/2700] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:28:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:28:10.636303 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121e4cdc-b2ce-4754-bff1-34bb9f9dcc7a admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:10.637869 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-121e4cdc-b2ce-4754-bff1-34bb9f9dcc7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:10.705111 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121e4cdc-b2ce-4754-bff1-34bb9f9dcc7a admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:10.709368 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb1989a-cd43-4ac6-b2ec-8c09991cc48e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:10.710099 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb1989a-cd43-4ac6-b2ec-8c09991cc48e service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:10.710483 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb1989a-cd43-4ac6-b2ec-8c09991cc48e service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:10.710845 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb1989a-cd43-4ac6-b2ec-8c09991cc48e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:10.720389 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb1989a-cd43-4ac6-b2ec-8c09991cc48e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:10.721874 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb1989a-cd43-4ac6-b2ec-8c09991cc48e service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:10.731531 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1346/2701] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:28:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:28:11.011678 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b395d3-a1bb-402a-a5ca-79e9608bbf83 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:11.013452 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a5b395d3-a1bb-402a-a5ca-79e9608bbf83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:11.026268 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b395d3-a1bb-402a-a5ca-79e9608bbf83 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:11.031466 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaf7366-d538-4d5e-93bc-f3d8ab09ed55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:11.032123 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf7366-d538-4d5e-93bc-f3d8ab09ed55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:11.032394 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf7366-d538-4d5e-93bc-f3d8ab09ed55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:11.032697 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf7366-d538-4d5e-93bc-f3d8ab09ed55 admin admin] PATH_INFO: `/v3/projects/e03d47527fff465b96b708eea9078cc7` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:11.034029 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaf7366-d538-4d5e-93bc-f3d8ab09ed55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e03d47527fff465b96b708eea9078cc7)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:11.036042 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaf7366-d538-4d5e-93bc-f3d8ab09ed55 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:11.094547 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-eeaf7366-d538-4d5e-93bc-f3d8ab09ed55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e03d47527fff465b96b708eea9078cc7', 'request_id': 'req-eeaf7366-d538-4d5e-93bc-f3d8ab09ed55'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:11.100493 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-eeaf7366-d538-4d5e-93bc-f3d8ab09ed55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e03d47527fff465b96b708eea9078cc7', 'request_id': 'req-eeaf7366-d538-4d5e-93bc-f3d8ab09ed55'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:11.108483 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-eeaf7366-d538-4d5e-93bc-f3d8ab09ed55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e03d47527fff465b96b708eea9078cc7', 'request_id': 'req-eeaf7366-d538-4d5e-93bc-f3d8ab09ed55'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:11.121339 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-eeaf7366-d538-4d5e-93bc-f3d8ab09ed55 admin admin] The token cache is being invalidate because project e03d47527fff465b96b708eea9078cc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:28:11.121680 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-eeaf7366-d538-4d5e-93bc-f3d8ab09ed55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:11.122499 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-eeaf7366-d538-4d5e-93bc-f3d8ab09ed55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:11.124140 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1356/2702] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:28:11 2022] DELETE /identity/v3/projects/e03d47527fff465b96b708eea9078cc7 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:28:11.128626 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb1989a-cd43-4ac6-b2ec-8c09991cc48e service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:11.139093 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb1989a-cd43-4ac6-b2ec-8c09991cc48e service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:11.158751 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:28:11.207912 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9937ef54-e594-423b-b3fc-a3d45cfa1f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:11.208655 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937ef54-e594-423b-b3fc-a3d45cfa1f84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:11.209164 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937ef54-e594-423b-b3fc-a3d45cfa1f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:11.209502 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937ef54-e594-423b-b3fc-a3d45cfa1f84 admin admin] PATH_INFO: `/v3/users/f6389340be944a4da816567f178eb2b2` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:11.210762 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9937ef54-e594-423b-b3fc-a3d45cfa1f84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6389340be944a4da816567f178eb2b2)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:11.212787 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9937ef54-e594-423b-b3fc-a3d45cfa1f84 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:11.260716 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-9937ef54-e594-423b-b3fc-a3d45cfa1f84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6389340be944a4da816567f178eb2b2'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:11.261460 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-9937ef54-e594-423b-b3fc-a3d45cfa1f84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6389340be944a4da816567f178eb2b2'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:11.269371 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-9937ef54-e594-423b-b3fc-a3d45cfa1f84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6389340be944a4da816567f178eb2b2'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:11.282444 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-9937ef54-e594-423b-b3fc-a3d45cfa1f84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6389340be944a4da816567f178eb2b2'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:11.291441 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-9937ef54-e594-423b-b3fc-a3d45cfa1f84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6389340be944a4da816567f178eb2b2'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:11.298720 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1347/2703] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:28:11 2022] DELETE /identity/v3/users/f6389340be944a4da816567f178eb2b2 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:28:11.576924 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaf7366-d538-4d5e-93bc-f3d8ab09ed55 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:11.578801 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-eeaf7366-d538-4d5e-93bc-f3d8ab09ed55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:11.597566 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaf7366-d538-4d5e-93bc-f3d8ab09ed55 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:11.662613 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4018980-5971-453e-91e7-0d182aff67d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:11.663263 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4018980-5971-453e-91e7-0d182aff67d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:11.663495 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4018980-5971-453e-91e7-0d182aff67d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:11.663774 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4018980-5971-453e-91e7-0d182aff67d3 admin admin] PATH_INFO: `/v3/projects/8061b0883ff4404b96e9e2173577f3b7` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:11.665024 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4018980-5971-453e-91e7-0d182aff67d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8061b0883ff4404b96e9e2173577f3b7)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:11.666963 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4018980-5971-453e-91e7-0d182aff67d3 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:11.701340 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:28:11.720356 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a4018980-5971-453e-91e7-0d182aff67d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8061b0883ff4404b96e9e2173577f3b7', 'request_id': 'req-a4018980-5971-453e-91e7-0d182aff67d3'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:11.726888 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a4018980-5971-453e-91e7-0d182aff67d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8061b0883ff4404b96e9e2173577f3b7', 'request_id': 'req-a4018980-5971-453e-91e7-0d182aff67d3'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:11.735174 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a4018980-5971-453e-91e7-0d182aff67d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8061b0883ff4404b96e9e2173577f3b7', 'request_id': 'req-a4018980-5971-453e-91e7-0d182aff67d3'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:11.746704 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a4018980-5971-453e-91e7-0d182aff67d3 admin admin] The token cache is being invalidate because project 8061b0883ff4404b96e9e2173577f3b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:28:11.746989 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a4018980-5971-453e-91e7-0d182aff67d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:11.747686 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a4018980-5971-453e-91e7-0d182aff67d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:11.749273 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1357/2704] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:28:11 2022] DELETE /identity/v3/projects/8061b0883ff4404b96e9e2173577f3b7 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:28:11.754408 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c362382-a674-458f-932b-97f6e99ca2e0 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:11.754909 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c362382-a674-458f-932b-97f6e99ca2e0 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:11.754909 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c362382-a674-458f-932b-97f6e99ca2e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:11.776086 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-8c362382-a674-458f-932b-97f6e99ca2e0 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:28:11.777262 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8c362382-a674-458f-932b-97f6e99ca2e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:11.838288 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1348/2705] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:28:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:28:11.846559 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb882253-9692-4531-95c9-db3f639ca252 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:11.846841 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb882253-9692-4531-95c9-db3f639ca252 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:11.847110 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb882253-9692-4531-95c9-db3f639ca252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:11.868952 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-fb882253-9692-4531-95c9-db3f639ca252 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:28:11.870198 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-fb882253-9692-4531-95c9-db3f639ca252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:11.878641 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1358/2706] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:28:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:28:11.883894 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c362382-a674-458f-932b-97f6e99ca2e0 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:11.894129 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c362382-a674-458f-932b-97f6e99ca2e0 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:11.899108 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14314523-3d49-4617-b12a-a73ffe1e380d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:11.899713 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14314523-3d49-4617-b12a-a73ffe1e380d admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:11.899998 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14314523-3d49-4617-b12a-a73ffe1e380d admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:11.900250 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14314523-3d49-4617-b12a-a73ffe1e380d admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:11.900929 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14314523-3d49-4617-b12a-a73ffe1e380d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:28:11.901229 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14314523-3d49-4617-b12a-a73ffe1e380d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:11.902838 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14314523-3d49-4617-b12a-a73ffe1e380d admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:11.914839 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1349/2707] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 6 16:28:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:28:11.919829 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb882253-9692-4531-95c9-db3f639ca252 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:11.930652 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb882253-9692-4531-95c9-db3f639ca252 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:11.935360 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1b05d7-048c-4d99-89dc-7fbf5da0f501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:11.935965 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1b05d7-048c-4d99-89dc-7fbf5da0f501 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:11.936242 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1b05d7-048c-4d99-89dc-7fbf5da0f501 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:11.936511 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1b05d7-048c-4d99-89dc-7fbf5da0f501 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:11.937212 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1b05d7-048c-4d99-89dc-7fbf5da0f501 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:11.939240 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1b05d7-048c-4d99-89dc-7fbf5da0f501 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:11.968297 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1359/2708] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:28:11 2022] POST /identity/v3/projects => generated 387 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:28:11.973223 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14314523-3d49-4617-b12a-a73ffe1e380d admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:11.984952 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14314523-3d49-4617-b12a-a73ffe1e380d admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:12.012742 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:28:12.047603 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1b05d7-048c-4d99-89dc-7fbf5da0f501 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:12.049893 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ed1b05d7-048c-4d99-89dc-7fbf5da0f501 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:12.053772 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c614326-346a-4aba-8d8d-bc68e2cba0b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:12.054579 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c614326-346a-4aba-8d8d-bc68e2cba0b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:12.054975 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c614326-346a-4aba-8d8d-bc68e2cba0b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:12.055320 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c614326-346a-4aba-8d8d-bc68e2cba0b1 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:12.056151 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c614326-346a-4aba-8d8d-bc68e2cba0b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:12.058354 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c614326-346a-4aba-8d8d-bc68e2cba0b1 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:12.092884 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1350/2709] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:28:11 2022] POST /identity/v3/users => generated 294 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:28:12.094552 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c614326-346a-4aba-8d8d-bc68e2cba0b1 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:12.109440 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c614326-346a-4aba-8d8d-bc68e2cba0b1 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:12.115266 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eae5b4-463e-40f8-8608-98740fe7a92a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:12.116245 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eae5b4-463e-40f8-8608-98740fe7a92a admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:12.116526 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eae5b4-463e-40f8-8608-98740fe7a92a admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:12.116827 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eae5b4-463e-40f8-8608-98740fe7a92a admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:12.117483 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eae5b4-463e-40f8-8608-98740fe7a92a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:12.119165 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eae5b4-463e-40f8-8608-98740fe7a92a admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:12.128101 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1b05d7-048c-4d99-89dc-7fbf5da0f501 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:12.130253 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1351/2710] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:28:12 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:28:12.132440 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb52728-1fa0-4f41-be0a-ddab098cd944 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:12.133183 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb52728-1fa0-4f41-be0a-ddab098cd944 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:12.133611 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb52728-1fa0-4f41-be0a-ddab098cd944 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:12.134054 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb52728-1fa0-4f41-be0a-ddab098cd944 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:12.134806 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eae5b4-463e-40f8-8608-98740fe7a92a admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:12.136894 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-aeb52728-1fa0-4f41-be0a-ddab098cd944 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:12.148892 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eae5b4-463e-40f8-8608-98740fe7a92a admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:12.153840 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea2a2ff-5924-4d81-a2b1-7739d17e8fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:12.154481 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2a2ff-5924-4d81-a2b1-7739d17e8fcc admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:12.154760 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2a2ff-5924-4d81-a2b1-7739d17e8fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:12.155030 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2a2ff-5924-4d81-a2b1-7739d17e8fcc admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:12.155711 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea2a2ff-5924-4d81-a2b1-7739d17e8fcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:12.157382 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea2a2ff-5924-4d81-a2b1-7739d17e8fcc admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:12.168504 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1352/2711] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:28:12 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:28:12.172961 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea2a2ff-5924-4d81-a2b1-7739d17e8fcc admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:12.183867 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea2a2ff-5924-4d81-a2b1-7739d17e8fcc admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:12.190925 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abde4560-c463-4d44-8eb6-1ea1c16ae998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:12.192028 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde4560-c463-4d44-8eb6-1ea1c16ae998 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:12.192523 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde4560-c463-4d44-8eb6-1ea1c16ae998 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:12.193004 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde4560-c463-4d44-8eb6-1ea1c16ae998 admin admin] PATH_INFO: `/v3/projects/787e58f3b25745e59a06677a85562dc8/users/5e5037b8b2a2456c92fd6ebe04a6e8b8/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:12.216324 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb52728-1fa0-4f41-be0a-ddab098cd944 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:12.216998 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abde4560-c463-4d44-8eb6-1ea1c16ae998 admin admin] RBAC: Authorizing `identity:create_grant(project_id=787e58f3b25745e59a06677a85562dc8, user_id=5e5037b8b2a2456c92fd6ebe04a6e8b8, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:12.218786 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb52728-1fa0-4f41-be0a-ddab098cd944 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:12.222243 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abde4560-c463-4d44-8eb6-1ea1c16ae998 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:12.236662 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1360/2712] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:28:12 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:28:12.244999 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1353/2713] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:28:12 2022] PUT /identity/v3/projects/787e58f3b25745e59a06677a85562dc8/users/5e5037b8b2a2456c92fd6ebe04a6e8b8/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:28:12.256558 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470a3545-2040-4ec2-94b0-e5ddfc3d810f None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:12.256868 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470a3545-2040-4ec2-94b0-e5ddfc3d810f None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:12.257145 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470a3545-2040-4ec2-94b0-e5ddfc3d810f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:12.287106 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-470a3545-2040-4ec2-94b0-e5ddfc3d810f None None] MFA Rules not processed for user `5e5037b8b2a2456c92fd6ebe04a6e8b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:28:12.288311 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-470a3545-2040-4ec2-94b0-e5ddfc3d810f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:12.323379 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:28:12.369797 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1361/2714] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:28:12 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:28:12.375244 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abde4560-c463-4d44-8eb6-1ea1c16ae998 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:12.459202 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abde4560-c463-4d44-8eb6-1ea1c16ae998 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:12.467312 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d728160-34cd-4fad-90d0-cd0d3673c8e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:12.468146 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d728160-34cd-4fad-90d0-cd0d3673c8e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:12.468488 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d728160-34cd-4fad-90d0-cd0d3673c8e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:12.469019 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d728160-34cd-4fad-90d0-cd0d3673c8e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:12.469775 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d728160-34cd-4fad-90d0-cd0d3673c8e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:12.472397 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d728160-34cd-4fad-90d0-cd0d3673c8e7 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:12.503386 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1354/2715] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:28:12 2022] POST /identity/v3/projects => generated 387 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:28:12.508115 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470a3545-2040-4ec2-94b0-e5ddfc3d810f None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:12.519065 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470a3545-2040-4ec2-94b0-e5ddfc3d810f None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:12.582911 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24ef76e-c29c-4a5a-a0af-49550acca590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:12.583609 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24ef76e-c29c-4a5a-a0af-49550acca590 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:12.583894 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24ef76e-c29c-4a5a-a0af-49550acca590 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:12.584258 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24ef76e-c29c-4a5a-a0af-49550acca590 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:12.584997 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24ef76e-c29c-4a5a-a0af-49550acca590 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:12.587368 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24ef76e-c29c-4a5a-a0af-49550acca590 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:12.611603 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1362/2716] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:28:12 2022] POST /identity/v3/users => generated 300 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:28:12.615294 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d728160-34cd-4fad-90d0-cd0d3673c8e7 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:12.626373 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d728160-34cd-4fad-90d0-cd0d3673c8e7 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:12.632375 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962715ab-2704-4bd4-9bfd-5a8e9b99da17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:12.633027 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962715ab-2704-4bd4-9bfd-5a8e9b99da17 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:12.633303 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962715ab-2704-4bd4-9bfd-5a8e9b99da17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:12.633568 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962715ab-2704-4bd4-9bfd-5a8e9b99da17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:12.634215 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962715ab-2704-4bd4-9bfd-5a8e9b99da17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:12.635829 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962715ab-2704-4bd4-9bfd-5a8e9b99da17 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:12.648641 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1355/2717] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:28:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:28:12.653553 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24ef76e-c29c-4a5a-a0af-49550acca590 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:12.666120 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24ef76e-c29c-4a5a-a0af-49550acca590 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:12.671836 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be91e846-4c61-499a-83e8-0b8da9a8f70d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:12.672506 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be91e846-4c61-499a-83e8-0b8da9a8f70d admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:12.672807 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be91e846-4c61-499a-83e8-0b8da9a8f70d admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:12.673181 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be91e846-4c61-499a-83e8-0b8da9a8f70d admin admin] PATH_INFO: `/v3/projects/d5e9c8dc710c41eaa8838748df75fede/users/3b985dc8f7c74c7dbfed93403c3e47dd/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:12.699108 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be91e846-4c61-499a-83e8-0b8da9a8f70d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5e9c8dc710c41eaa8838748df75fede, user_id=3b985dc8f7c74c7dbfed93403c3e47dd, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:12.701838 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be91e846-4c61-499a-83e8-0b8da9a8f70d admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:12.715518 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1363/2718] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:28:12 2022] PUT /identity/v3/projects/d5e9c8dc710c41eaa8838748df75fede/users/3b985dc8f7c74c7dbfed93403c3e47dd/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:28:12.719838 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962715ab-2704-4bd4-9bfd-5a8e9b99da17 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:12.729895 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962715ab-2704-4bd4-9bfd-5a8e9b99da17 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:12.774055 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:28:12.788015 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52afc913-26b8-4c70-8f01-704f9b6c67cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:12.788629 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52afc913-26b8-4c70-8f01-704f9b6c67cd admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:12.788937 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52afc913-26b8-4c70-8f01-704f9b6c67cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:12.789207 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52afc913-26b8-4c70-8f01-704f9b6c67cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:12.789890 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52afc913-26b8-4c70-8f01-704f9b6c67cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:12.791499 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52afc913-26b8-4c70-8f01-704f9b6c67cd admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:12.802043 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1356/2719] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:28:12 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:28:12.806593 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be91e846-4c61-499a-83e8-0b8da9a8f70d admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:12.817160 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be91e846-4c61-499a-83e8-0b8da9a8f70d admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:12.822322 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c27f12-44f3-4566-a191-9a0c0d8adc03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:12.823031 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c27f12-44f3-4566-a191-9a0c0d8adc03 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:12.823310 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c27f12-44f3-4566-a191-9a0c0d8adc03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:12.823585 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c27f12-44f3-4566-a191-9a0c0d8adc03 admin admin] PATH_INFO: `/v3/projects/d5e9c8dc710c41eaa8838748df75fede/users/3b985dc8f7c74c7dbfed93403c3e47dd/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:12.825641 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c27f12-44f3-4566-a191-9a0c0d8adc03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5e9c8dc710c41eaa8838748df75fede, user_id=3b985dc8f7c74c7dbfed93403c3e47dd, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:12.828443 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c27f12-44f3-4566-a191-9a0c0d8adc03 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:12.841203 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1364/2720] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:28:12 2022] PUT /identity/v3/projects/d5e9c8dc710c41eaa8838748df75fede/users/3b985dc8f7c74c7dbfed93403c3e47dd/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:28:12.851738 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b4b81a-520f-425b-9efe-c4eaddfb58e9 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:12.852023 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b4b81a-520f-425b-9efe-c4eaddfb58e9 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:12.852291 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b4b81a-520f-425b-9efe-c4eaddfb58e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:12.882025 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-d7b4b81a-520f-425b-9efe-c4eaddfb58e9 None None] MFA Rules not processed for user `3b985dc8f7c74c7dbfed93403c3e47dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:28:12.883458 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d7b4b81a-520f-425b-9efe-c4eaddfb58e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:12.970116 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1357/2721] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:28:12 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:28:12.982458 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c27f12-44f3-4566-a191-9a0c0d8adc03 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:13.044203 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c27f12-44f3-4566-a191-9a0c0d8adc03 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:13.048438 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a394ed-5114-447a-bc8a-9c334069938f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:13.049197 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a394ed-5114-447a-bc8a-9c334069938f service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:13.049827 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a394ed-5114-447a-bc8a-9c334069938f service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:13.050099 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a394ed-5114-447a-bc8a-9c334069938f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:13.060389 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a394ed-5114-447a-bc8a-9c334069938f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:13.061892 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a394ed-5114-447a-bc8a-9c334069938f service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:13.071617 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1365/2722] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:28:12 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:28:13.100879 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b4b81a-520f-425b-9efe-c4eaddfb58e9 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:13.169278 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b4b81a-520f-425b-9efe-c4eaddfb58e9 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:13.174680 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642e329b-7176-4f0e-9a2b-dd483be465e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:13.175372 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642e329b-7176-4f0e-9a2b-dd483be465e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:13.175651 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642e329b-7176-4f0e-9a2b-dd483be465e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:13.175928 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642e329b-7176-4f0e-9a2b-dd483be465e7 admin admin] PATH_INFO: `/v3/users/5e5037b8b2a2456c92fd6ebe04a6e8b8` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:13.177225 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642e329b-7176-4f0e-9a2b-dd483be465e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e5037b8b2a2456c92fd6ebe04a6e8b8)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:13.179242 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642e329b-7176-4f0e-9a2b-dd483be465e7 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:13.228842 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-642e329b-7176-4f0e-9a2b-dd483be465e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e5037b8b2a2456c92fd6ebe04a6e8b8'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:13.229539 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-642e329b-7176-4f0e-9a2b-dd483be465e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e5037b8b2a2456c92fd6ebe04a6e8b8'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:13.236695 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-642e329b-7176-4f0e-9a2b-dd483be465e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e5037b8b2a2456c92fd6ebe04a6e8b8'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:13.252910 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:28:13.256533 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-642e329b-7176-4f0e-9a2b-dd483be465e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e5037b8b2a2456c92fd6ebe04a6e8b8'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:13.265927 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-642e329b-7176-4f0e-9a2b-dd483be465e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e5037b8b2a2456c92fd6ebe04a6e8b8'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:13.272715 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1358/2723] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:28:13 2022] DELETE /identity/v3/users/5e5037b8b2a2456c92fd6ebe04a6e8b8 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:28:13.277763 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3798ffdf-48fb-4f40-b1f6-059c6d25af65 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:13.278058 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3798ffdf-48fb-4f40-b1f6-059c6d25af65 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:13.278447 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3798ffdf-48fb-4f40-b1f6-059c6d25af65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:13.300664 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-3798ffdf-48fb-4f40-b1f6-059c6d25af65 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:28:13.301903 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3798ffdf-48fb-4f40-b1f6-059c6d25af65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:13.323155 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:28:13.373983 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1366/2724] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:28:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:28:13.384113 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642e329b-7176-4f0e-9a2b-dd483be465e7 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:13.385820 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-642e329b-7176-4f0e-9a2b-dd483be465e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:13.450976 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642e329b-7176-4f0e-9a2b-dd483be465e7 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:13.454772 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6020ad-6b02-4881-b545-745f816a1736 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:13.455426 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6020ad-6b02-4881-b545-745f816a1736 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:13.455704 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6020ad-6b02-4881-b545-745f816a1736 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:13.455974 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6020ad-6b02-4881-b545-745f816a1736 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:13.465455 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6020ad-6b02-4881-b545-745f816a1736 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:13.466970 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6020ad-6b02-4881-b545-745f816a1736 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:13.478377 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1359/2725] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:28:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:28:13.752511 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3798ffdf-48fb-4f40-b1f6-059c6d25af65 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:13.754099 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3798ffdf-48fb-4f40-b1f6-059c6d25af65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:13.767939 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3798ffdf-48fb-4f40-b1f6-059c6d25af65 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:13.774823 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afb01fe-59a8-4d2f-9944-6d13d0c297f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:13.775598 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb01fe-59a8-4d2f-9944-6d13d0c297f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:13.775773 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb01fe-59a8-4d2f-9944-6d13d0c297f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:13.775993 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb01fe-59a8-4d2f-9944-6d13d0c297f6 admin admin] PATH_INFO: `/v3/projects/787e58f3b25745e59a06677a85562dc8` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:13.777254 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afb01fe-59a8-4d2f-9944-6d13d0c297f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=787e58f3b25745e59a06677a85562dc8)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:13.779192 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afb01fe-59a8-4d2f-9944-6d13d0c297f6 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:13.826218 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8afb01fe-59a8-4d2f-9944-6d13d0c297f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '787e58f3b25745e59a06677a85562dc8', 'request_id': 'req-8afb01fe-59a8-4d2f-9944-6d13d0c297f6'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:13.831577 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8afb01fe-59a8-4d2f-9944-6d13d0c297f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '787e58f3b25745e59a06677a85562dc8', 'request_id': 'req-8afb01fe-59a8-4d2f-9944-6d13d0c297f6'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:13.840549 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8afb01fe-59a8-4d2f-9944-6d13d0c297f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '787e58f3b25745e59a06677a85562dc8', 'request_id': 'req-8afb01fe-59a8-4d2f-9944-6d13d0c297f6'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:13.852797 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8afb01fe-59a8-4d2f-9944-6d13d0c297f6 admin admin] The token cache is being invalidate because project 787e58f3b25745e59a06677a85562dc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:28:13.853103 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8afb01fe-59a8-4d2f-9944-6d13d0c297f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:13.853844 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8afb01fe-59a8-4d2f-9944-6d13d0c297f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:13.855454 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1367/2726] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:28:13 2022] DELETE /identity/v3/projects/787e58f3b25745e59a06677a85562dc8 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:28:13.859716 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6020ad-6b02-4881-b545-745f816a1736 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:13.869720 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6020ad-6b02-4881-b545-745f816a1736 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:13.929578 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd1b3cc-2523-45ac-89c3-cf158c0b1cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:13.930120 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd1b3cc-2523-45ac-89c3-cf158c0b1cf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:13.930395 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd1b3cc-2523-45ac-89c3-cf158c0b1cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:13.930663 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd1b3cc-2523-45ac-89c3-cf158c0b1cf8 admin admin] PATH_INFO: `/v3/users/3b985dc8f7c74c7dbfed93403c3e47dd` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:13.931795 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd1b3cc-2523-45ac-89c3-cf158c0b1cf8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b985dc8f7c74c7dbfed93403c3e47dd)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:13.933775 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd1b3cc-2523-45ac-89c3-cf158c0b1cf8 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:13.982690 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-edd1b3cc-2523-45ac-89c3-cf158c0b1cf8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b985dc8f7c74c7dbfed93403c3e47dd'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:13.983349 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-edd1b3cc-2523-45ac-89c3-cf158c0b1cf8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b985dc8f7c74c7dbfed93403c3e47dd'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:13.990221 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-edd1b3cc-2523-45ac-89c3-cf158c0b1cf8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b985dc8f7c74c7dbfed93403c3e47dd'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:14.003029 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-edd1b3cc-2523-45ac-89c3-cf158c0b1cf8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b985dc8f7c74c7dbfed93403c3e47dd'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:14.010680 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-edd1b3cc-2523-45ac-89c3-cf158c0b1cf8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b985dc8f7c74c7dbfed93403c3e47dd'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:14.016770 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1360/2727] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:28:13 2022] DELETE /identity/v3/users/3b985dc8f7c74c7dbfed93403c3e47dd => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:28:14.295409 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afb01fe-59a8-4d2f-9944-6d13d0c297f6 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:14.297037 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-8afb01fe-59a8-4d2f-9944-6d13d0c297f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:14.313513 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afb01fe-59a8-4d2f-9944-6d13d0c297f6 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:14.373940 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20246a6-ea48-4ced-8238-a2a52216c529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:14.374587 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20246a6-ea48-4ced-8238-a2a52216c529 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:14.374867 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20246a6-ea48-4ced-8238-a2a52216c529 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:14.375141 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20246a6-ea48-4ced-8238-a2a52216c529 admin admin] PATH_INFO: `/v3/projects/d5e9c8dc710c41eaa8838748df75fede` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:14.376372 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20246a6-ea48-4ced-8238-a2a52216c529 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5e9c8dc710c41eaa8838748df75fede)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:14.378634 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20246a6-ea48-4ced-8238-a2a52216c529 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:14.431017 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-f20246a6-ea48-4ced-8238-a2a52216c529 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5e9c8dc710c41eaa8838748df75fede', 'request_id': 'req-f20246a6-ea48-4ced-8238-a2a52216c529'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:14.432356 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:28:14.438335 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-f20246a6-ea48-4ced-8238-a2a52216c529 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5e9c8dc710c41eaa8838748df75fede', 'request_id': 'req-f20246a6-ea48-4ced-8238-a2a52216c529'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:14.446842 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-f20246a6-ea48-4ced-8238-a2a52216c529 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5e9c8dc710c41eaa8838748df75fede', 'request_id': 'req-f20246a6-ea48-4ced-8238-a2a52216c529'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:14.458612 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-f20246a6-ea48-4ced-8238-a2a52216c529 admin admin] The token cache is being invalidate because project d5e9c8dc710c41eaa8838748df75fede was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:28:14.458901 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-f20246a6-ea48-4ced-8238-a2a52216c529 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:14.459621 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-f20246a6-ea48-4ced-8238-a2a52216c529 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:14.461270 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1368/2728] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:28:14 2022] DELETE /identity/v3/projects/d5e9c8dc710c41eaa8838748df75fede => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:28:14.466418 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba2e727-82df-46be-91a1-e7a5376bb2f3 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:14.466640 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba2e727-82df-46be-91a1-e7a5376bb2f3 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:14.466915 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba2e727-82df-46be-91a1-e7a5376bb2f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:14.487783 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-0ba2e727-82df-46be-91a1-e7a5376bb2f3 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:28:14.489050 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0ba2e727-82df-46be-91a1-e7a5376bb2f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:14.493837 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:28:14.556058 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1361/2729] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:28:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:28:14.564409 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cd3d45-b116-4a65-8a53-178ff78f336e None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:14.564788 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cd3d45-b116-4a65-8a53-178ff78f336e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:14.565067 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cd3d45-b116-4a65-8a53-178ff78f336e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:14.585808 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-22cd3d45-b116-4a65-8a53-178ff78f336e None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:28:14.587007 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-22cd3d45-b116-4a65-8a53-178ff78f336e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:14.595440 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1369/2730] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:28:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:28:14.600035 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba2e727-82df-46be-91a1-e7a5376bb2f3 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:14.610829 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba2e727-82df-46be-91a1-e7a5376bb2f3 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:14.619187 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7c9dff-7b13-444e-b71f-489108c62242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:14.619857 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7c9dff-7b13-444e-b71f-489108c62242 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:14.620306 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7c9dff-7b13-444e-b71f-489108c62242 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:14.620583 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7c9dff-7b13-444e-b71f-489108c62242 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:14.621275 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7c9dff-7b13-444e-b71f-489108c62242 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:28:14.621580 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7c9dff-7b13-444e-b71f-489108c62242 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:14.623226 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7c9dff-7b13-444e-b71f-489108c62242 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:14.637119 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1362/2731] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 6 16:28:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:28:14.642166 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cd3d45-b116-4a65-8a53-178ff78f336e None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:14.654107 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cd3d45-b116-4a65-8a53-178ff78f336e None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:14.660469 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b793ae3e-b0ed-4217-9037-641614838d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:14.661142 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b793ae3e-b0ed-4217-9037-641614838d08 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:14.661415 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b793ae3e-b0ed-4217-9037-641614838d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:14.661678 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b793ae3e-b0ed-4217-9037-641614838d08 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:14.662309 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b793ae3e-b0ed-4217-9037-641614838d08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:14.664350 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b793ae3e-b0ed-4217-9037-641614838d08 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:14.697825 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1370/2732] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:28:14 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:28:14.702197 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7c9dff-7b13-444e-b71f-489108c62242 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:14.713220 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7c9dff-7b13-444e-b71f-489108c62242 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:14.779115 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad60780-30de-45e6-92b9-04ebb87fc24c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:14.779727 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad60780-30de-45e6-92b9-04ebb87fc24c admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:14.780001 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad60780-30de-45e6-92b9-04ebb87fc24c admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:14.780277 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad60780-30de-45e6-92b9-04ebb87fc24c admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:14.780939 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad60780-30de-45e6-92b9-04ebb87fc24c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:14.782859 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad60780-30de-45e6-92b9-04ebb87fc24c admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:14.805715 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1363/2733] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:28:14 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:28:14.810198 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b793ae3e-b0ed-4217-9037-641614838d08 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:14.821512 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b793ae3e-b0ed-4217-9037-641614838d08 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:14.826833 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c09a024-3aa1-45fc-a5ea-f8a873392795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:14.827435 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c09a024-3aa1-45fc-a5ea-f8a873392795 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:14.827706 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c09a024-3aa1-45fc-a5ea-f8a873392795 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:14.827968 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c09a024-3aa1-45fc-a5ea-f8a873392795 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:14.828604 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c09a024-3aa1-45fc-a5ea-f8a873392795 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:14.830265 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c09a024-3aa1-45fc-a5ea-f8a873392795 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:14.841019 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1371/2734] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:28:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:28:14.845301 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad60780-30de-45e6-92b9-04ebb87fc24c admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:14.856116 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad60780-30de-45e6-92b9-04ebb87fc24c admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:14.861953 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e98ea26-7c80-4f17-9228-dcb50e1324f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:14.862551 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e98ea26-7c80-4f17-9228-dcb50e1324f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:14.862829 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e98ea26-7c80-4f17-9228-dcb50e1324f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:14.863086 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e98ea26-7c80-4f17-9228-dcb50e1324f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:14.863730 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e98ea26-7c80-4f17-9228-dcb50e1324f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:14.865367 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e98ea26-7c80-4f17-9228-dcb50e1324f8 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:14.876058 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1364/2735] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:28:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:28:14.881307 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c09a024-3aa1-45fc-a5ea-f8a873392795 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:14.892743 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c09a024-3aa1-45fc-a5ea-f8a873392795 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:14.898062 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a516e34d-b898-4c8b-8d10-23a2bc67fdf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:14.898764 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a516e34d-b898-4c8b-8d10-23a2bc67fdf3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:14.899044 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a516e34d-b898-4c8b-8d10-23a2bc67fdf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:14.899319 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a516e34d-b898-4c8b-8d10-23a2bc67fdf3 admin admin] PATH_INFO: `/v3/projects/b97a86135d3541e1b8e07177379ad0e0/users/e65899e16060404e806b657e18894756/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:14.915360 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a516e34d-b898-4c8b-8d10-23a2bc67fdf3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b97a86135d3541e1b8e07177379ad0e0, user_id=e65899e16060404e806b657e18894756, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:14.918261 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a516e34d-b898-4c8b-8d10-23a2bc67fdf3 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:14.937095 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1372/2736] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:28:14 2022] PUT /identity/v3/projects/b97a86135d3541e1b8e07177379ad0e0/users/e65899e16060404e806b657e18894756/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:28:15.012399 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff489ee9-ed66-41f7-9060-fe8f2d96eff2 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:15.012687 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff489ee9-ed66-41f7-9060-fe8f2d96eff2 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:15.012981 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff489ee9-ed66-41f7-9060-fe8f2d96eff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:15.042203 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-ff489ee9-ed66-41f7-9060-fe8f2d96eff2 None None] MFA Rules not processed for user `e65899e16060404e806b657e18894756`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:28:15.043361 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ff489ee9-ed66-41f7-9060-fe8f2d96eff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:15.111308 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1365/2737] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:28:15 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:28:15.116183 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a516e34d-b898-4c8b-8d10-23a2bc67fdf3 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:15.179785 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a516e34d-b898-4c8b-8d10-23a2bc67fdf3 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:15.184914 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ec9f8e-940f-4028-9e4b-fb8579248d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:15.185533 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ec9f8e-940f-4028-9e4b-fb8579248d4b admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:15.185834 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ec9f8e-940f-4028-9e4b-fb8579248d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:15.186106 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ec9f8e-940f-4028-9e4b-fb8579248d4b admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:15.186737 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ec9f8e-940f-4028-9e4b-fb8579248d4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:15.188666 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ec9f8e-940f-4028-9e4b-fb8579248d4b admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:15.217338 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1373/2738] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:28:15 2022] POST /identity/v3/projects => generated 387 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:28:15.222032 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff489ee9-ed66-41f7-9060-fe8f2d96eff2 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:15.229403 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:28:15.235115 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff489ee9-ed66-41f7-9060-fe8f2d96eff2 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:15.305029 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12a6e15-1eb4-416f-b721-c933fc48fbb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:15.305556 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12a6e15-1eb4-416f-b721-c933fc48fbb7 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:15.305831 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12a6e15-1eb4-416f-b721-c933fc48fbb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:15.306097 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12a6e15-1eb4-416f-b721-c933fc48fbb7 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:15.306735 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12a6e15-1eb4-416f-b721-c933fc48fbb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:15.308696 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12a6e15-1eb4-416f-b721-c933fc48fbb7 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:15.331064 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1366/2739] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:28:15 2022] POST /identity/v3/users => generated 294 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:28:15.335104 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ec9f8e-940f-4028-9e4b-fb8579248d4b admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:15.346790 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ec9f8e-940f-4028-9e4b-fb8579248d4b admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:15.351844 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d51d093-3b1a-47e7-8a3f-47d470e93e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:15.352474 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d51d093-3b1a-47e7-8a3f-47d470e93e5f admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:15.352780 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d51d093-3b1a-47e7-8a3f-47d470e93e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:15.353046 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d51d093-3b1a-47e7-8a3f-47d470e93e5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:15.353686 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d51d093-3b1a-47e7-8a3f-47d470e93e5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:15.355295 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d51d093-3b1a-47e7-8a3f-47d470e93e5f admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:15.366595 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1374/2740] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:28:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 16:28:15.370750 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12a6e15-1eb4-416f-b721-c933fc48fbb7 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:15.381419 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12a6e15-1eb4-416f-b721-c933fc48fbb7 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:15.386293 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437cad58-7fca-4b87-a557-72ae6e6f4867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:15.386889 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437cad58-7fca-4b87-a557-72ae6e6f4867 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:15.387167 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437cad58-7fca-4b87-a557-72ae6e6f4867 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:15.387449 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437cad58-7fca-4b87-a557-72ae6e6f4867 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:15.388086 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437cad58-7fca-4b87-a557-72ae6e6f4867 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:15.389726 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437cad58-7fca-4b87-a557-72ae6e6f4867 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:15.399850 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1367/2741] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:28:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 16:28:15.404426 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d51d093-3b1a-47e7-8a3f-47d470e93e5f admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:15.415287 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d51d093-3b1a-47e7-8a3f-47d470e93e5f admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:15.420735 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f379b45e-00ec-4508-ab51-1bd6ce6025c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:15.421422 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f379b45e-00ec-4508-ab51-1bd6ce6025c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:15.421691 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f379b45e-00ec-4508-ab51-1bd6ce6025c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:15.421954 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f379b45e-00ec-4508-ab51-1bd6ce6025c2 admin admin] PATH_INFO: `/v3/projects/2f2c4c950b6f422e836df819c2623be7/users/660d13ef13184ef0ae1210948f045d0a/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:15.437554 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f379b45e-00ec-4508-ab51-1bd6ce6025c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f2c4c950b6f422e836df819c2623be7, user_id=660d13ef13184ef0ae1210948f045d0a, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:15.440336 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f379b45e-00ec-4508-ab51-1bd6ce6025c2 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:15.454275 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1375/2742] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:28:15 2022] PUT /identity/v3/projects/2f2c4c950b6f422e836df819c2623be7/users/660d13ef13184ef0ae1210948f045d0a/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:28:15.464730 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64310e6-3ad4-4311-aa3d-140a67407b78 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:15.465041 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64310e6-3ad4-4311-aa3d-140a67407b78 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:15.465308 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64310e6-3ad4-4311-aa3d-140a67407b78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:15.496208 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-c64310e6-3ad4-4311-aa3d-140a67407b78 None None] MFA Rules not processed for user `660d13ef13184ef0ae1210948f045d0a`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:28:15.497463 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c64310e6-3ad4-4311-aa3d-140a67407b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:15.566093 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1368/2743] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:28:15 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:28:15.571405 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f379b45e-00ec-4508-ab51-1bd6ce6025c2 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:15.604622 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:DELETE=3,keystone:SELECT=92,keystone:INSERT=5 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:28:15.653634 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f379b45e-00ec-4508-ab51-1bd6ce6025c2 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:15.660239 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778f453f-f40d-4b48-8d40-cbce2bf5e591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:15.660671 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778f453f-f40d-4b48-8d40-cbce2bf5e591 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:15.660956 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778f453f-f40d-4b48-8d40-cbce2bf5e591 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:15.661253 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778f453f-f40d-4b48-8d40-cbce2bf5e591 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:15.661886 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778f453f-f40d-4b48-8d40-cbce2bf5e591 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:15.663900 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778f453f-f40d-4b48-8d40-cbce2bf5e591 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:15.698953 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1376/2744] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:28:15 2022] POST /identity/v3/projects => generated 387 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:28:15.700235 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64310e6-3ad4-4311-aa3d-140a67407b78 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:15.712778 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64310e6-3ad4-4311-aa3d-140a67407b78 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:15.773768 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e806cf-5e8a-4e47-b03f-fff8660b41b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:15.774395 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e806cf-5e8a-4e47-b03f-fff8660b41b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:15.774673 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e806cf-5e8a-4e47-b03f-fff8660b41b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:15.774941 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e806cf-5e8a-4e47-b03f-fff8660b41b6 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:15.775694 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e806cf-5e8a-4e47-b03f-fff8660b41b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:15.777663 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e806cf-5e8a-4e47-b03f-fff8660b41b6 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:15.799810 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1369/2745] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:28:15 2022] POST /identity/v3/users => generated 300 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:28:15.804000 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778f453f-f40d-4b48-8d40-cbce2bf5e591 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:15.815085 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778f453f-f40d-4b48-8d40-cbce2bf5e591 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:15.820637 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21a4149-40cb-4680-9e96-4d6ee8071491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:15.821243 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a4149-40cb-4680-9e96-4d6ee8071491 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:15.821507 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a4149-40cb-4680-9e96-4d6ee8071491 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:15.821783 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a4149-40cb-4680-9e96-4d6ee8071491 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:15.822445 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21a4149-40cb-4680-9e96-4d6ee8071491 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:15.824065 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21a4149-40cb-4680-9e96-4d6ee8071491 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:15.835341 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1377/2746] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:28:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:28:15.839686 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e806cf-5e8a-4e47-b03f-fff8660b41b6 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:15.845655 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:28:15.853434 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e806cf-5e8a-4e47-b03f-fff8660b41b6 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:15.860454 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa51cc43-ffec-426d-b4a9-cff5e621adb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:15.861425 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51cc43-ffec-426d-b4a9-cff5e621adb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:15.861745 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51cc43-ffec-426d-b4a9-cff5e621adb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:15.862211 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51cc43-ffec-426d-b4a9-cff5e621adb3 admin admin] PATH_INFO: `/v3/projects/99fec40c681c4c9eabb5a1074e1cb028/users/ed7c80a289514084a654de53fa4572aa/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:15.880850 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa51cc43-ffec-426d-b4a9-cff5e621adb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99fec40c681c4c9eabb5a1074e1cb028, user_id=ed7c80a289514084a654de53fa4572aa, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:15.883735 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa51cc43-ffec-426d-b4a9-cff5e621adb3 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:15.898155 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1370/2747] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:28:15 2022] PUT /identity/v3/projects/99fec40c681c4c9eabb5a1074e1cb028/users/ed7c80a289514084a654de53fa4572aa/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:28:15.902414 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21a4149-40cb-4680-9e96-4d6ee8071491 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:15.912726 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21a4149-40cb-4680-9e96-4d6ee8071491 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:15.975876 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a358e3b3-c08b-43d2-afe8-48b0ca3aad96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:15.976484 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a358e3b3-c08b-43d2-afe8-48b0ca3aad96 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:15.976791 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a358e3b3-c08b-43d2-afe8-48b0ca3aad96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:15.977092 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a358e3b3-c08b-43d2-afe8-48b0ca3aad96 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:15.977722 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a358e3b3-c08b-43d2-afe8-48b0ca3aad96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:15.979329 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a358e3b3-c08b-43d2-afe8-48b0ca3aad96 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:15.989762 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1378/2748] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:28:15 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:28:15.995058 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa51cc43-ffec-426d-b4a9-cff5e621adb3 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:16.011098 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa51cc43-ffec-426d-b4a9-cff5e621adb3 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:16.016990 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eca28da-8acc-426e-9b0e-9eaf7619784b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:16.017859 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eca28da-8acc-426e-9b0e-9eaf7619784b admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:16.018161 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eca28da-8acc-426e-9b0e-9eaf7619784b admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:16.018433 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eca28da-8acc-426e-9b0e-9eaf7619784b admin admin] PATH_INFO: `/v3/projects/99fec40c681c4c9eabb5a1074e1cb028/users/ed7c80a289514084a654de53fa4572aa/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:16.020991 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eca28da-8acc-426e-9b0e-9eaf7619784b admin admin] RBAC: Authorizing `identity:create_grant(project_id=99fec40c681c4c9eabb5a1074e1cb028, user_id=ed7c80a289514084a654de53fa4572aa, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:16.023708 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eca28da-8acc-426e-9b0e-9eaf7619784b admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:16.035107 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1371/2749] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:28:15 2022] PUT /identity/v3/projects/99fec40c681c4c9eabb5a1074e1cb028/users/ed7c80a289514084a654de53fa4572aa/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:28:16.046891 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f4d17-daf4-4da2-a924-45a8da71f0d0 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:16.047177 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f4d17-daf4-4da2-a924-45a8da71f0d0 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:16.047450 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f4d17-daf4-4da2-a924-45a8da71f0d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:16.077454 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-489f4d17-daf4-4da2-a924-45a8da71f0d0 None None] MFA Rules not processed for user `ed7c80a289514084a654de53fa4572aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:28:16.078589 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-489f4d17-daf4-4da2-a924-45a8da71f0d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:16.153591 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1379/2750] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:28:16 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:28:16.164818 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eca28da-8acc-426e-9b0e-9eaf7619784b admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:16.166767 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7eca28da-8acc-426e-9b0e-9eaf7619784b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:16.228710 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eca28da-8acc-426e-9b0e-9eaf7619784b admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:16.232799 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf90c8f-539f-4d49-a93b-ec5710690e1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:16.233733 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf90c8f-539f-4d49-a93b-ec5710690e1c service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:16.234162 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf90c8f-539f-4d49-a93b-ec5710690e1c service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:16.234574 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf90c8f-539f-4d49-a93b-ec5710690e1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:16.249085 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf90c8f-539f-4d49-a93b-ec5710690e1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:16.250577 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf90c8f-539f-4d49-a93b-ec5710690e1c service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:16.268685 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1372/2751] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:28:16 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:28:16.810052 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489f4d17-daf4-4da2-a924-45a8da71f0d0 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:16.821560 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489f4d17-daf4-4da2-a924-45a8da71f0d0 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:16.825169 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1610060d-834f-4477-90c2-f4da7458d1af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:16.825783 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1610060d-834f-4477-90c2-f4da7458d1af service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:16.826077 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1610060d-834f-4477-90c2-f4da7458d1af service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:16.826349 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1610060d-834f-4477-90c2-f4da7458d1af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:16.844227 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:28:16.888793 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1610060d-834f-4477-90c2-f4da7458d1af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:16.890300 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1610060d-834f-4477-90c2-f4da7458d1af service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:16.901691 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1380/2752] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:28:16 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:28:23.595783 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41376773-d579-49f0-ada1-28470eb85a32 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:23.596273 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41376773-d579-49f0-ada1-28470eb85a32 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:23.596273 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41376773-d579-49f0-ada1-28470eb85a32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:23.616423 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-41376773-d579-49f0-ada1-28470eb85a32 None None] MFA Rules not processed for user `4d3856b7bb7f4f5bb801952b893bbe35`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:28:23.617657 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-41376773-d579-49f0-ada1-28470eb85a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:23.676963 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1373/2753] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:28:23 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:28:26.297178 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1610060d-834f-4477-90c2-f4da7458d1af service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:26.307274 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1610060d-834f-4477-90c2-f4da7458d1af service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:26.313352 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79276b4-393e-4408-8e9c-0db60908a5f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:26.313977 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79276b4-393e-4408-8e9c-0db60908a5f2 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:26.314253 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79276b4-393e-4408-8e9c-0db60908a5f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:26.314519 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79276b4-393e-4408-8e9c-0db60908a5f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:26.379817 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79276b4-393e-4408-8e9c-0db60908a5f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:26.381458 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79276b4-393e-4408-8e9c-0db60908a5f2 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:26.390589 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1381/2754] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:28:26 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:28:28.712585 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41376773-d579-49f0-ada1-28470eb85a32 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:28.777776 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41376773-d579-49f0-ada1-28470eb85a32 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:28.782881 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34be3d6d-187d-41fb-a29b-ca68b063548a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:28.783520 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be3d6d-187d-41fb-a29b-ca68b063548a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:28.783891 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be3d6d-187d-41fb-a29b-ca68b063548a admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:28.784096 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be3d6d-187d-41fb-a29b-ca68b063548a admin admin] PATH_INFO: `/v3/users/e65899e16060404e806b657e18894756` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:28.785507 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34be3d6d-187d-41fb-a29b-ca68b063548a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e65899e16060404e806b657e18894756)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:28.788006 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34be3d6d-187d-41fb-a29b-ca68b063548a admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:28.806855 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:28:28.843754 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-34be3d6d-187d-41fb-a29b-ca68b063548a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e65899e16060404e806b657e18894756'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:28.844473 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-34be3d6d-187d-41fb-a29b-ca68b063548a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e65899e16060404e806b657e18894756'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:28.852213 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-34be3d6d-187d-41fb-a29b-ca68b063548a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e65899e16060404e806b657e18894756'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:28.869075 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-34be3d6d-187d-41fb-a29b-ca68b063548a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e65899e16060404e806b657e18894756'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:28.879365 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-34be3d6d-187d-41fb-a29b-ca68b063548a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e65899e16060404e806b657e18894756'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:28.888698 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1374/2755] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:28:28 2022] DELETE /identity/v3/users/e65899e16060404e806b657e18894756 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:28:28.893844 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0add27d2-83e6-4024-8fb0-a8cfd3299983 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:28.894389 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0add27d2-83e6-4024-8fb0-a8cfd3299983 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:28.894507 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0add27d2-83e6-4024-8fb0-a8cfd3299983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:28.915514 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-0add27d2-83e6-4024-8fb0-a8cfd3299983 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:28:28.916735 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0add27d2-83e6-4024-8fb0-a8cfd3299983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:28.983344 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1382/2756] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:28:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:28:28.993297 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34be3d6d-187d-41fb-a29b-ca68b063548a admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:28.994797 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-34be3d6d-187d-41fb-a29b-ca68b063548a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:29.056164 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34be3d6d-187d-41fb-a29b-ca68b063548a admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:29.060171 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94e689f-8748-4117-a446-170af957f8dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:29.060796 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94e689f-8748-4117-a446-170af957f8dc service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:29.061103 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94e689f-8748-4117-a446-170af957f8dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:29.061364 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94e689f-8748-4117-a446-170af957f8dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:29.071669 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94e689f-8748-4117-a446-170af957f8dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:29.073212 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94e689f-8748-4117-a446-170af957f8dc service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:29.083675 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1375/2757] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:28:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:28:29.681855 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0add27d2-83e6-4024-8fb0-a8cfd3299983 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:29.683472 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0add27d2-83e6-4024-8fb0-a8cfd3299983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:29.696291 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0add27d2-83e6-4024-8fb0-a8cfd3299983 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:29.707584 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0b560b-a654-47c5-8f0b-45b0df8f861c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:29.708260 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0b560b-a654-47c5-8f0b-45b0df8f861c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:29.708542 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0b560b-a654-47c5-8f0b-45b0df8f861c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:29.708858 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0b560b-a654-47c5-8f0b-45b0df8f861c admin admin] PATH_INFO: `/v3/projects/b97a86135d3541e1b8e07177379ad0e0` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:29.710177 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0b560b-a654-47c5-8f0b-45b0df8f861c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b97a86135d3541e1b8e07177379ad0e0)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:29.712133 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0b560b-a654-47c5-8f0b-45b0df8f861c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:29.762342 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-fa0b560b-a654-47c5-8f0b-45b0df8f861c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b97a86135d3541e1b8e07177379ad0e0', 'request_id': 'req-fa0b560b-a654-47c5-8f0b-45b0df8f861c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:29.764416 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:28:29.770133 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-fa0b560b-a654-47c5-8f0b-45b0df8f861c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b97a86135d3541e1b8e07177379ad0e0', 'request_id': 'req-fa0b560b-a654-47c5-8f0b-45b0df8f861c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:29.778351 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-fa0b560b-a654-47c5-8f0b-45b0df8f861c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b97a86135d3541e1b8e07177379ad0e0', 'request_id': 'req-fa0b560b-a654-47c5-8f0b-45b0df8f861c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:29.790707 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-fa0b560b-a654-47c5-8f0b-45b0df8f861c admin admin] The token cache is being invalidate because project b97a86135d3541e1b8e07177379ad0e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:28:29.791006 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-fa0b560b-a654-47c5-8f0b-45b0df8f861c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:29.791630 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-fa0b560b-a654-47c5-8f0b-45b0df8f861c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:29.793165 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1383/2758] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:28:29 2022] DELETE /identity/v3/projects/b97a86135d3541e1b8e07177379ad0e0 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:28:29.797509 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94e689f-8748-4117-a446-170af957f8dc service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:29.807959 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94e689f-8748-4117-a446-170af957f8dc service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:29.877590 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1460f5b3-1034-42b9-9965-d1272dd96c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:29.878223 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1460f5b3-1034-42b9-9965-d1272dd96c83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:29.878497 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1460f5b3-1034-42b9-9965-d1272dd96c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:29.878763 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1460f5b3-1034-42b9-9965-d1272dd96c83 admin admin] PATH_INFO: `/v3/users/660d13ef13184ef0ae1210948f045d0a` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:29.880047 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1460f5b3-1034-42b9-9965-d1272dd96c83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=660d13ef13184ef0ae1210948f045d0a)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:29.882091 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1460f5b3-1034-42b9-9965-d1272dd96c83 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:29.921217 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:28:29.931475 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1460f5b3-1034-42b9-9965-d1272dd96c83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '660d13ef13184ef0ae1210948f045d0a'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:29.932376 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1460f5b3-1034-42b9-9965-d1272dd96c83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '660d13ef13184ef0ae1210948f045d0a'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:29.939097 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1460f5b3-1034-42b9-9965-d1272dd96c83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '660d13ef13184ef0ae1210948f045d0a'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:29.952164 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1460f5b3-1034-42b9-9965-d1272dd96c83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '660d13ef13184ef0ae1210948f045d0a'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:29.961676 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1460f5b3-1034-42b9-9965-d1272dd96c83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '660d13ef13184ef0ae1210948f045d0a'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:29.968509 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1376/2759] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:28:29 2022] DELETE /identity/v3/users/660d13ef13184ef0ae1210948f045d0a => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:28:30.259490 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0b560b-a654-47c5-8f0b-45b0df8f861c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:30.261370 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-fa0b560b-a654-47c5-8f0b-45b0df8f861c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:30.283736 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0b560b-a654-47c5-8f0b-45b0df8f861c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:30.350653 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5edce70-2f17-42c8-94d3-339dec28af1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:30.351297 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5edce70-2f17-42c8-94d3-339dec28af1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:30.351575 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5edce70-2f17-42c8-94d3-339dec28af1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:30.351845 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5edce70-2f17-42c8-94d3-339dec28af1e admin admin] PATH_INFO: `/v3/projects/2f2c4c950b6f422e836df819c2623be7` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:30.353015 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5edce70-2f17-42c8-94d3-339dec28af1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f2c4c950b6f422e836df819c2623be7)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:30.354954 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5edce70-2f17-42c8-94d3-339dec28af1e admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:30.408401 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-c5edce70-2f17-42c8-94d3-339dec28af1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f2c4c950b6f422e836df819c2623be7', 'request_id': 'req-c5edce70-2f17-42c8-94d3-339dec28af1e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:30.414780 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-c5edce70-2f17-42c8-94d3-339dec28af1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f2c4c950b6f422e836df819c2623be7', 'request_id': 'req-c5edce70-2f17-42c8-94d3-339dec28af1e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:30.424953 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-c5edce70-2f17-42c8-94d3-339dec28af1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f2c4c950b6f422e836df819c2623be7', 'request_id': 'req-c5edce70-2f17-42c8-94d3-339dec28af1e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:30.436643 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-c5edce70-2f17-42c8-94d3-339dec28af1e admin admin] The token cache is being invalidate because project 2f2c4c950b6f422e836df819c2623be7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:28:30.437029 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-c5edce70-2f17-42c8-94d3-339dec28af1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:30.437694 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-c5edce70-2f17-42c8-94d3-339dec28af1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:30.439238 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1384/2760] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:28:30 2022] DELETE /identity/v3/projects/2f2c4c950b6f422e836df819c2623be7 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:28:30.443624 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1460f5b3-1034-42b9-9965-d1272dd96c83 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:30.445238 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-1460f5b3-1034-42b9-9965-d1272dd96c83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:30.459055 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1460f5b3-1034-42b9-9965-d1272dd96c83 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:30.535617 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a205907-4d94-4314-919f-8cbc2562b4fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:30.536380 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a205907-4d94-4314-919f-8cbc2562b4fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:30.536661 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a205907-4d94-4314-919f-8cbc2562b4fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:30.536970 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a205907-4d94-4314-919f-8cbc2562b4fb admin admin] PATH_INFO: `/v3/users/ed7c80a289514084a654de53fa4572aa` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:30.538215 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a205907-4d94-4314-919f-8cbc2562b4fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed7c80a289514084a654de53fa4572aa)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:30.540220 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a205907-4d94-4314-919f-8cbc2562b4fb admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:30.589771 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-4a205907-4d94-4314-919f-8cbc2562b4fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed7c80a289514084a654de53fa4572aa'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:30.590440 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-4a205907-4d94-4314-919f-8cbc2562b4fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed7c80a289514084a654de53fa4572aa'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:30.597556 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-4a205907-4d94-4314-919f-8cbc2562b4fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed7c80a289514084a654de53fa4572aa'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:30.609799 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-4a205907-4d94-4314-919f-8cbc2562b4fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed7c80a289514084a654de53fa4572aa'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:30.618348 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-4a205907-4d94-4314-919f-8cbc2562b4fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed7c80a289514084a654de53fa4572aa'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:30.625060 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1377/2761] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:28:30 2022] DELETE /identity/v3/users/ed7c80a289514084a654de53fa4572aa => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:28:30.915641 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5edce70-2f17-42c8-94d3-339dec28af1e admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:30.917229 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-c5edce70-2f17-42c8-94d3-339dec28af1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:30.932179 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5edce70-2f17-42c8-94d3-339dec28af1e admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:30.992415 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2723f8e5-35f8-43db-92b8-d4337d32e325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:30.993089 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2723f8e5-35f8-43db-92b8-d4337d32e325 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:30.993363 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2723f8e5-35f8-43db-92b8-d4337d32e325 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:30.993630 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2723f8e5-35f8-43db-92b8-d4337d32e325 admin admin] PATH_INFO: `/v3/projects/99fec40c681c4c9eabb5a1074e1cb028` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:30.994760 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2723f8e5-35f8-43db-92b8-d4337d32e325 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99fec40c681c4c9eabb5a1074e1cb028)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:30.996699 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2723f8e5-35f8-43db-92b8-d4337d32e325 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:31.008592 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:28:31.044006 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2723f8e5-35f8-43db-92b8-d4337d32e325 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99fec40c681c4c9eabb5a1074e1cb028', 'request_id': 'req-2723f8e5-35f8-43db-92b8-d4337d32e325'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:31.049963 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2723f8e5-35f8-43db-92b8-d4337d32e325 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99fec40c681c4c9eabb5a1074e1cb028', 'request_id': 'req-2723f8e5-35f8-43db-92b8-d4337d32e325'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:31.058006 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2723f8e5-35f8-43db-92b8-d4337d32e325 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99fec40c681c4c9eabb5a1074e1cb028', 'request_id': 'req-2723f8e5-35f8-43db-92b8-d4337d32e325'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:31.069716 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2723f8e5-35f8-43db-92b8-d4337d32e325 admin admin] The token cache is being invalidate because project 99fec40c681c4c9eabb5a1074e1cb028 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:28:31.070001 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2723f8e5-35f8-43db-92b8-d4337d32e325 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:31.070645 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2723f8e5-35f8-43db-92b8-d4337d32e325 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:31.072148 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1385/2762] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:28:30 2022] DELETE /identity/v3/projects/99fec40c681c4c9eabb5a1074e1cb028 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:28:31.079496 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7503387-afe6-41c5-81e1-d1cdc2a01202 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:31.080019 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7503387-afe6-41c5-81e1-d1cdc2a01202 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:31.080019 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7503387-afe6-41c5-81e1-d1cdc2a01202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:31.101449 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-e7503387-afe6-41c5-81e1-d1cdc2a01202 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:28:31.102631 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e7503387-afe6-41c5-81e1-d1cdc2a01202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:31.137041 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:28:31.171697 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1378/2763] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:28:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:28:31.180916 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b465c6-b00b-405f-8d57-7e693f1756f9 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:31.181207 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b465c6-b00b-405f-8d57-7e693f1756f9 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:31.181481 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b465c6-b00b-405f-8d57-7e693f1756f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:31.203713 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-84b465c6-b00b-405f-8d57-7e693f1756f9 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:28:31.204951 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-84b465c6-b00b-405f-8d57-7e693f1756f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:31.215105 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1386/2764] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:28:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:28:31.219550 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7503387-afe6-41c5-81e1-d1cdc2a01202 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:31.231170 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7503387-afe6-41c5-81e1-d1cdc2a01202 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:31.236981 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9b6740-5810-4a14-b11f-d06ad3b53bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:31.237564 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9b6740-5810-4a14-b11f-d06ad3b53bf3 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:31.237840 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9b6740-5810-4a14-b11f-d06ad3b53bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:31.238104 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9b6740-5810-4a14-b11f-d06ad3b53bf3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:31.238755 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9b6740-5810-4a14-b11f-d06ad3b53bf3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:28:31.239052 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9b6740-5810-4a14-b11f-d06ad3b53bf3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:31.240722 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9b6740-5810-4a14-b11f-d06ad3b53bf3 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:31.253790 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1379/2765] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 6 16:28:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:28:31.258452 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b465c6-b00b-405f-8d57-7e693f1756f9 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:31.270797 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b465c6-b00b-405f-8d57-7e693f1756f9 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:31.276245 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ed7422-2118-47f6-84d5-633639c70ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:31.276882 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ed7422-2118-47f6-84d5-633639c70ec5 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:31.277157 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ed7422-2118-47f6-84d5-633639c70ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:31.277405 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ed7422-2118-47f6-84d5-633639c70ec5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:31.278047 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ed7422-2118-47f6-84d5-633639c70ec5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:31.280719 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ed7422-2118-47f6-84d5-633639c70ec5 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:31.312956 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1387/2766] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:28:31 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:28:31.317683 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9b6740-5810-4a14-b11f-d06ad3b53bf3 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:31.328168 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9b6740-5810-4a14-b11f-d06ad3b53bf3 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:31.387885 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f39bc0-6594-4b9b-8f45-95f928be6b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:31.388540 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f39bc0-6594-4b9b-8f45-95f928be6b0e admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:31.388860 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f39bc0-6594-4b9b-8f45-95f928be6b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:31.389130 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f39bc0-6594-4b9b-8f45-95f928be6b0e admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:31.389758 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f39bc0-6594-4b9b-8f45-95f928be6b0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:31.391668 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f39bc0-6594-4b9b-8f45-95f928be6b0e admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:31.416619 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1380/2767] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:28:31 2022] POST /identity/v3/users => generated 287 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:28:31.421653 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ed7422-2118-47f6-84d5-633639c70ec5 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:31.435101 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ed7422-2118-47f6-84d5-633639c70ec5 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:31.440426 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae90a63a-8a62-4034-a8ef-05c417d36886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:31.441046 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae90a63a-8a62-4034-a8ef-05c417d36886 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:31.441325 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae90a63a-8a62-4034-a8ef-05c417d36886 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:31.441588 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae90a63a-8a62-4034-a8ef-05c417d36886 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:31.442228 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae90a63a-8a62-4034-a8ef-05c417d36886 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:31.443851 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae90a63a-8a62-4034-a8ef-05c417d36886 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:31.453853 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1388/2768] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:28:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:28:31.460326 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f39bc0-6594-4b9b-8f45-95f928be6b0e admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:31.470613 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f39bc0-6594-4b9b-8f45-95f928be6b0e admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:31.475695 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46578fc-610c-410f-9330-86012efc3b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:31.476314 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46578fc-610c-410f-9330-86012efc3b88 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:31.476592 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46578fc-610c-410f-9330-86012efc3b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:31.476899 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46578fc-610c-410f-9330-86012efc3b88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:31.477557 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46578fc-610c-410f-9330-86012efc3b88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:31.479178 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46578fc-610c-410f-9330-86012efc3b88 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:31.491167 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1381/2769] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:28:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:28:31.496016 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae90a63a-8a62-4034-a8ef-05c417d36886 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:31.510663 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae90a63a-8a62-4034-a8ef-05c417d36886 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:31.518908 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acce34a0-8952-441d-91d6-a4cabeb14abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:31.519879 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acce34a0-8952-441d-91d6-a4cabeb14abb admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:31.520162 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acce34a0-8952-441d-91d6-a4cabeb14abb admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:31.520454 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acce34a0-8952-441d-91d6-a4cabeb14abb admin admin] PATH_INFO: `/v3/projects/640d35cd7b4c47e4a696269f86b4db53/users/eab3fce828a5402caf4b93eeeb06c5e1/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:31.537825 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acce34a0-8952-441d-91d6-a4cabeb14abb admin admin] RBAC: Authorizing `identity:create_grant(project_id=640d35cd7b4c47e4a696269f86b4db53, user_id=eab3fce828a5402caf4b93eeeb06c5e1, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:31.540489 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acce34a0-8952-441d-91d6-a4cabeb14abb admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:31.555704 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1389/2770] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:28:31 2022] PUT /identity/v3/projects/640d35cd7b4c47e4a696269f86b4db53/users/eab3fce828a5402caf4b93eeeb06c5e1/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:28:31.567854 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21f6989-8e0b-4751-b5b6-cd71d48b27a1 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:31.568137 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21f6989-8e0b-4751-b5b6-cd71d48b27a1 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:31.568408 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21f6989-8e0b-4751-b5b6-cd71d48b27a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:31.597750 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-d21f6989-8e0b-4751-b5b6-cd71d48b27a1 None None] MFA Rules not processed for user `eab3fce828a5402caf4b93eeeb06c5e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:28:31.598984 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d21f6989-8e0b-4751-b5b6-cd71d48b27a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:31.666037 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1382/2771] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:28:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:28:31.678273 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acce34a0-8952-441d-91d6-a4cabeb14abb admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:31.680183 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-acce34a0-8952-441d-91d6-a4cabeb14abb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:31.742818 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acce34a0-8952-441d-91d6-a4cabeb14abb admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:31.746677 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f66b25d-fc34-498e-8727-53c2a3cdde51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:31.747314 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66b25d-fc34-498e-8727-53c2a3cdde51 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:31.747589 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66b25d-fc34-498e-8727-53c2a3cdde51 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:31.747855 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66b25d-fc34-498e-8727-53c2a3cdde51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:31.756476 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f66b25d-fc34-498e-8727-53c2a3cdde51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:31.757986 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f66b25d-fc34-498e-8727-53c2a3cdde51 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:31.766914 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1390/2772] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:28:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:28:34.844391 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21f6989-8e0b-4751-b5b6-cd71d48b27a1 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:34.884625 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:28:34.919714 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21f6989-8e0b-4751-b5b6-cd71d48b27a1 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:34.924608 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a17b9dd-63bf-498f-9086-2094d9ba5ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:34.925262 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17b9dd-63bf-498f-9086-2094d9ba5ad6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:34.925535 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17b9dd-63bf-498f-9086-2094d9ba5ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:34.925802 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17b9dd-63bf-498f-9086-2094d9ba5ad6 admin admin] PATH_INFO: `/v3/users/eab3fce828a5402caf4b93eeeb06c5e1` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:34.926915 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a17b9dd-63bf-498f-9086-2094d9ba5ad6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eab3fce828a5402caf4b93eeeb06c5e1)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:34.928872 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a17b9dd-63bf-498f-9086-2094d9ba5ad6 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:34.981626 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1a17b9dd-63bf-498f-9086-2094d9ba5ad6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eab3fce828a5402caf4b93eeeb06c5e1'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:34.982290 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1a17b9dd-63bf-498f-9086-2094d9ba5ad6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eab3fce828a5402caf4b93eeeb06c5e1'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:34.989133 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1a17b9dd-63bf-498f-9086-2094d9ba5ad6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eab3fce828a5402caf4b93eeeb06c5e1'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:35.001298 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1a17b9dd-63bf-498f-9086-2094d9ba5ad6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eab3fce828a5402caf4b93eeeb06c5e1'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:35.020620 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1a17b9dd-63bf-498f-9086-2094d9ba5ad6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eab3fce828a5402caf4b93eeeb06c5e1'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:35.027006 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1383/2773] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:28:34 2022] DELETE /identity/v3/users/eab3fce828a5402caf4b93eeeb06c5e1 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:28:35.031896 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc394923-be22-414a-8725-e02b452b59b2 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:35.032432 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc394923-be22-414a-8725-e02b452b59b2 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:35.032561 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc394923-be22-414a-8725-e02b452b59b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:35.053133 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-cc394923-be22-414a-8725-e02b452b59b2 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:28:35.055805 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-cc394923-be22-414a-8725-e02b452b59b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:35.088254 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:28:35.117307 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1391/2774] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:28:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:28:35.127705 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a17b9dd-63bf-498f-9086-2094d9ba5ad6 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:35.129532 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-1a17b9dd-63bf-498f-9086-2094d9ba5ad6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:35.196295 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a17b9dd-63bf-498f-9086-2094d9ba5ad6 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:35.200679 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb75db7-d119-4043-8876-370b722be1b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:35.201595 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb75db7-d119-4043-8876-370b722be1b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:35.201841 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb75db7-d119-4043-8876-370b722be1b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:35.202118 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb75db7-d119-4043-8876-370b722be1b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:35.213056 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb75db7-d119-4043-8876-370b722be1b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:35.214486 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb75db7-d119-4043-8876-370b722be1b6 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:35.225168 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1384/2775] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:28:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:28:35.528544 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc394923-be22-414a-8725-e02b452b59b2 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:35.530019 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-cc394923-be22-414a-8725-e02b452b59b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:35.542305 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc394923-be22-414a-8725-e02b452b59b2 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:35.547815 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ded0fc4-cfab-43a9-8ec6-2cae335824ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:35.548453 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ded0fc4-cfab-43a9-8ec6-2cae335824ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:35.548717 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ded0fc4-cfab-43a9-8ec6-2cae335824ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:35.549367 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ded0fc4-cfab-43a9-8ec6-2cae335824ac admin admin] PATH_INFO: `/v3/projects/640d35cd7b4c47e4a696269f86b4db53` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:35.550630 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ded0fc4-cfab-43a9-8ec6-2cae335824ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=640d35cd7b4c47e4a696269f86b4db53)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:35.552605 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ded0fc4-cfab-43a9-8ec6-2cae335824ac admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:35.615490 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5ded0fc4-cfab-43a9-8ec6-2cae335824ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '640d35cd7b4c47e4a696269f86b4db53', 'request_id': 'req-5ded0fc4-cfab-43a9-8ec6-2cae335824ac'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:35.624138 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5ded0fc4-cfab-43a9-8ec6-2cae335824ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '640d35cd7b4c47e4a696269f86b4db53', 'request_id': 'req-5ded0fc4-cfab-43a9-8ec6-2cae335824ac'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:35.634130 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5ded0fc4-cfab-43a9-8ec6-2cae335824ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '640d35cd7b4c47e4a696269f86b4db53', 'request_id': 'req-5ded0fc4-cfab-43a9-8ec6-2cae335824ac'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:35.646628 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5ded0fc4-cfab-43a9-8ec6-2cae335824ac admin admin] The token cache is being invalidate because project 640d35cd7b4c47e4a696269f86b4db53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:28:35.646964 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5ded0fc4-cfab-43a9-8ec6-2cae335824ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:35.647606 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5ded0fc4-cfab-43a9-8ec6-2cae335824ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:35.649218 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1392/2776] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:28:35 2022] DELETE /identity/v3/projects/640d35cd7b4c47e4a696269f86b4db53 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:28:35.654341 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7209811f-3ff1-44a4-96c6-05d39972830b None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:35.654627 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7209811f-3ff1-44a4-96c6-05d39972830b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:35.654901 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7209811f-3ff1-44a4-96c6-05d39972830b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:35.675865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-7209811f-3ff1-44a4-96c6-05d39972830b None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:28:35.677115 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7209811f-3ff1-44a4-96c6-05d39972830b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:35.725678 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:28:35.742052 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1385/2777] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:28:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:28:35.751212 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f327c6e2-d5ad-40ed-9b7e-cfd424c01aa6 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:35.751489 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f327c6e2-d5ad-40ed-9b7e-cfd424c01aa6 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:35.751760 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f327c6e2-d5ad-40ed-9b7e-cfd424c01aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:35.776323 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-f327c6e2-d5ad-40ed-9b7e-cfd424c01aa6 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:28:35.777638 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f327c6e2-d5ad-40ed-9b7e-cfd424c01aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:35.786553 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1393/2778] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:28:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:28:35.790922 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7209811f-3ff1-44a4-96c6-05d39972830b None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:35.801494 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7209811f-3ff1-44a4-96c6-05d39972830b None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:35.806606 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e50f7a6-99e6-470d-93ae-de5d0d194201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:35.807229 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e50f7a6-99e6-470d-93ae-de5d0d194201 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:35.807500 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e50f7a6-99e6-470d-93ae-de5d0d194201 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:35.807799 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e50f7a6-99e6-470d-93ae-de5d0d194201 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:35.808436 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e50f7a6-99e6-470d-93ae-de5d0d194201 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:28:35.808733 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e50f7a6-99e6-470d-93ae-de5d0d194201 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:35.810390 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e50f7a6-99e6-470d-93ae-de5d0d194201 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:35.823976 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1386/2779] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 6 16:28:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:28:35.827490 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f327c6e2-d5ad-40ed-9b7e-cfd424c01aa6 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:35.839500 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f327c6e2-d5ad-40ed-9b7e-cfd424c01aa6 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:35.848169 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbdad2b-84d2-4fbf-b526-4bf217e81364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:35.849005 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbdad2b-84d2-4fbf-b526-4bf217e81364 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:35.849005 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbdad2b-84d2-4fbf-b526-4bf217e81364 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:35.849327 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbdad2b-84d2-4fbf-b526-4bf217e81364 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:35.850189 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbdad2b-84d2-4fbf-b526-4bf217e81364 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:35.853161 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbdad2b-84d2-4fbf-b526-4bf217e81364 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:35.887335 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1394/2780] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:28:35 2022] POST /identity/v3/projects => generated 373 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:28:35.892324 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e50f7a6-99e6-470d-93ae-de5d0d194201 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:35.905628 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e50f7a6-99e6-470d-93ae-de5d0d194201 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:35.967065 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52831840-d147-4782-b8d6-77bd939a43ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:35.967975 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52831840-d147-4782-b8d6-77bd939a43ce admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:35.968396 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52831840-d147-4782-b8d6-77bd939a43ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:35.968810 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52831840-d147-4782-b8d6-77bd939a43ce admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:35.969759 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52831840-d147-4782-b8d6-77bd939a43ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:35.972303 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52831840-d147-4782-b8d6-77bd939a43ce admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:36.002612 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1387/2781] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:28:35 2022] POST /identity/v3/users => generated 287 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:28:36.007028 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbdad2b-84d2-4fbf-b526-4bf217e81364 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:36.018934 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbdad2b-84d2-4fbf-b526-4bf217e81364 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:36.024538 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d80048d-8d61-4ecb-9f99-1b4d3828ee83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:36.025184 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80048d-8d61-4ecb-9f99-1b4d3828ee83 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:36.025460 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80048d-8d61-4ecb-9f99-1b4d3828ee83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:36.025728 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80048d-8d61-4ecb-9f99-1b4d3828ee83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:36.026373 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d80048d-8d61-4ecb-9f99-1b4d3828ee83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:36.027988 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d80048d-8d61-4ecb-9f99-1b4d3828ee83 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:36.038282 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1395/2782] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:28:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:28:36.042385 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52831840-d147-4782-b8d6-77bd939a43ce admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:36.052877 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52831840-d147-4782-b8d6-77bd939a43ce admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:36.058104 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c4ae94-ad0c-4799-89e5-2f3c2e4629d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:36.058676 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c4ae94-ad0c-4799-89e5-2f3c2e4629d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:36.058951 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c4ae94-ad0c-4799-89e5-2f3c2e4629d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:36.059220 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c4ae94-ad0c-4799-89e5-2f3c2e4629d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:36.059850 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c4ae94-ad0c-4799-89e5-2f3c2e4629d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:36.061462 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c4ae94-ad0c-4799-89e5-2f3c2e4629d7 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:36.071998 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1388/2783] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:28:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:28:36.076213 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d80048d-8d61-4ecb-9f99-1b4d3828ee83 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:36.086556 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d80048d-8d61-4ecb-9f99-1b4d3828ee83 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:36.092176 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608e5c87-100a-4eea-810b-e627c629c5cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:36.092859 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608e5c87-100a-4eea-810b-e627c629c5cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:36.093140 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608e5c87-100a-4eea-810b-e627c629c5cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:36.093410 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608e5c87-100a-4eea-810b-e627c629c5cf admin admin] PATH_INFO: `/v3/projects/6e4dab6b2e7f407a814d1b042ad0dc2e/users/aea153fd2ffe4d728490be70e92d1896/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:36.109952 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608e5c87-100a-4eea-810b-e627c629c5cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e4dab6b2e7f407a814d1b042ad0dc2e, user_id=aea153fd2ffe4d728490be70e92d1896, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:36.112652 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608e5c87-100a-4eea-810b-e627c629c5cf admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:36.130167 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1396/2784] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:28:36 2022] PUT /identity/v3/projects/6e4dab6b2e7f407a814d1b042ad0dc2e/users/aea153fd2ffe4d728490be70e92d1896/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:28:36.140824 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb15306c-551a-4b38-b91b-8b9927c27a04 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:36.141057 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb15306c-551a-4b38-b91b-8b9927c27a04 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:36.141352 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb15306c-551a-4b38-b91b-8b9927c27a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:36.170637 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-bb15306c-551a-4b38-b91b-8b9927c27a04 None None] MFA Rules not processed for user `aea153fd2ffe4d728490be70e92d1896`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:28:36.171861 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-bb15306c-551a-4b38-b91b-8b9927c27a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:36.242069 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1389/2785] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:28:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:28:36.256470 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608e5c87-100a-4eea-810b-e627c629c5cf admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:36.259354 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-608e5c87-100a-4eea-810b-e627c629c5cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:36.319349 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608e5c87-100a-4eea-810b-e627c629c5cf admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:36.323442 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be9c173-0a08-4cd0-879a-f6677da983c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:36.324048 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9c173-0a08-4cd0-879a-f6677da983c7 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:36.324329 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9c173-0a08-4cd0-879a-f6677da983c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:36.324603 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9c173-0a08-4cd0-879a-f6677da983c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:36.333364 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be9c173-0a08-4cd0-879a-f6677da983c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:36.334823 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be9c173-0a08-4cd0-879a-f6677da983c7 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:36.346571 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1397/2786] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:28:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:28:36.375727 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb15306c-551a-4b38-b91b-8b9927c27a04 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:36.434764 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:28:36.449223 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb15306c-551a-4b38-b91b-8b9927c27a04 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:36.455434 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49313924-9ed5-4f2c-a3fb-d553952462be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:36.456082 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49313924-9ed5-4f2c-a3fb-d553952462be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:36.456355 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49313924-9ed5-4f2c-a3fb-d553952462be admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:36.456621 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49313924-9ed5-4f2c-a3fb-d553952462be admin admin] PATH_INFO: `/v3/users/aea153fd2ffe4d728490be70e92d1896` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:36.457793 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49313924-9ed5-4f2c-a3fb-d553952462be admin admin] RBAC: Authorizing `identity:delete_user(user_id=aea153fd2ffe4d728490be70e92d1896)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:36.459740 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49313924-9ed5-4f2c-a3fb-d553952462be admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:36.509442 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-49313924-9ed5-4f2c-a3fb-d553952462be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aea153fd2ffe4d728490be70e92d1896'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:36.510036 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-49313924-9ed5-4f2c-a3fb-d553952462be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aea153fd2ffe4d728490be70e92d1896'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:36.517336 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-49313924-9ed5-4f2c-a3fb-d553952462be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aea153fd2ffe4d728490be70e92d1896'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:36.529106 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-49313924-9ed5-4f2c-a3fb-d553952462be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aea153fd2ffe4d728490be70e92d1896'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:36.539697 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-49313924-9ed5-4f2c-a3fb-d553952462be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aea153fd2ffe4d728490be70e92d1896'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:36.547303 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1390/2787] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:28:36 2022] DELETE /identity/v3/users/aea153fd2ffe4d728490be70e92d1896 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:28:36.551775 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecf5b44-e989-4607-8013-d68a72541b87 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:36.552064 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecf5b44-e989-4607-8013-d68a72541b87 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:36.552366 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecf5b44-e989-4607-8013-d68a72541b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:36.559711 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:28:36.581506 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-4ecf5b44-e989-4607-8013-d68a72541b87 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:28:36.582823 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-4ecf5b44-e989-4607-8013-d68a72541b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:36.641544 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1398/2788] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:28:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:28:36.651764 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49313924-9ed5-4f2c-a3fb-d553952462be admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:36.653343 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-49313924-9ed5-4f2c-a3fb-d553952462be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:36.714547 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49313924-9ed5-4f2c-a3fb-d553952462be admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:36.718360 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2b5363-801b-446c-80a7-228a8cf65096 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:36.718961 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2b5363-801b-446c-80a7-228a8cf65096 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:36.719232 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2b5363-801b-446c-80a7-228a8cf65096 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:36.719496 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2b5363-801b-446c-80a7-228a8cf65096 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:36.729606 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2b5363-801b-446c-80a7-228a8cf65096 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:36.731838 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2b5363-801b-446c-80a7-228a8cf65096 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:36.742100 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1391/2789] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:28:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:28:37.027615 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecf5b44-e989-4607-8013-d68a72541b87 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:37.029253 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-4ecf5b44-e989-4607-8013-d68a72541b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:37.044606 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecf5b44-e989-4607-8013-d68a72541b87 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:37.049782 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af0c93b-46f5-48c8-abbf-2827b799b70f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:37.050418 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af0c93b-46f5-48c8-abbf-2827b799b70f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:37.050696 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af0c93b-46f5-48c8-abbf-2827b799b70f admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:37.050962 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af0c93b-46f5-48c8-abbf-2827b799b70f admin admin] PATH_INFO: `/v3/projects/6e4dab6b2e7f407a814d1b042ad0dc2e` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:37.052126 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af0c93b-46f5-48c8-abbf-2827b799b70f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e4dab6b2e7f407a814d1b042ad0dc2e)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:37.054187 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af0c93b-46f5-48c8-abbf-2827b799b70f admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:37.101969 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-1af0c93b-46f5-48c8-abbf-2827b799b70f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e4dab6b2e7f407a814d1b042ad0dc2e', 'request_id': 'req-1af0c93b-46f5-48c8-abbf-2827b799b70f'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:37.108252 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-1af0c93b-46f5-48c8-abbf-2827b799b70f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e4dab6b2e7f407a814d1b042ad0dc2e', 'request_id': 'req-1af0c93b-46f5-48c8-abbf-2827b799b70f'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:37.116151 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-1af0c93b-46f5-48c8-abbf-2827b799b70f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e4dab6b2e7f407a814d1b042ad0dc2e', 'request_id': 'req-1af0c93b-46f5-48c8-abbf-2827b799b70f'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:37.130442 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-1af0c93b-46f5-48c8-abbf-2827b799b70f admin admin] The token cache is being invalidate because project 6e4dab6b2e7f407a814d1b042ad0dc2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:28:37.130754 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-1af0c93b-46f5-48c8-abbf-2827b799b70f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:37.131494 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-1af0c93b-46f5-48c8-abbf-2827b799b70f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:37.135253 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1399/2790] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:28:37 2022] DELETE /identity/v3/projects/6e4dab6b2e7f407a814d1b042ad0dc2e => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:28:41.556360 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2b5363-801b-446c-80a7-228a8cf65096 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:41.566157 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2b5363-801b-446c-80a7-228a8cf65096 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:41.622182 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb81eca-c456-4131-9d76-14711faf95af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:41.622861 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb81eca-c456-4131-9d76-14711faf95af service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:41.623140 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb81eca-c456-4131-9d76-14711faf95af service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:41.623409 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb81eca-c456-4131-9d76-14711faf95af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:41.625325 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-2eb81eca-c456-4131-9d76-14711faf95af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:41.652209 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:28:41.700943 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb81eca-c456-4131-9d76-14711faf95af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:41.702389 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb81eca-c456-4131-9d76-14711faf95af service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:41.720074 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1392/2791] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:28:41 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:28:43.484939 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af0c93b-46f5-48c8-abbf-2827b799b70f admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:43.486612 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1af0c93b-46f5-48c8-abbf-2827b799b70f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:43.566160 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af0c93b-46f5-48c8-abbf-2827b799b70f admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:43.572520 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf51a2c-4855-464f-8f32-43df2505d7a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63de41f4348243998029f7d07aa5dc40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:43.573184 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf51a2c-4855-464f-8f32-43df2505d7a0 service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:43.573460 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf51a2c-4855-464f-8f32-43df2505d7a0 service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:43.573724 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf51a2c-4855-464f-8f32-43df2505d7a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:43.575679 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-bcf51a2c-4855-464f-8f32-43df2505d7a0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:43.589111 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf51a2c-4855-464f-8f32-43df2505d7a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:43.591149 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf51a2c-4855-464f-8f32-43df2505d7a0 service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:43.602822 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1400/2792] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:28:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:28:51.714868 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=19 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:28:53.599154 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=87,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:29:39.530982 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405b31db-b0c8-432d-8682-a6c84035f4e5 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:29:39.531647 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405b31db-b0c8-432d-8682-a6c84035f4e5 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:29:39.531647 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405b31db-b0c8-432d-8682-a6c84035f4e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:29:39.558119 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-405b31db-b0c8-432d-8682-a6c84035f4e5 None None] MFA Rules not processed for user `523faa1ac64e4044b9fb651ec28249d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:29:39.559382 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-405b31db-b0c8-432d-8682-a6c84035f4e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:29:39.622110 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1393/2793] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:29:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:29:39.971448 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf51a2c-4855-464f-8f32-43df2505d7a0 service nova] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:29:39.973787 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-bcf51a2c-4855-464f-8f32-43df2505d7a0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:29:39.990455 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf51a2c-4855-464f-8f32-43df2505d7a0 service nova] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:29:39.994914 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de218ae-1541-4d10-a48f-262aa27e1c7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:29:39.995531 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de218ae-1541-4d10-a48f-262aa27e1c7c service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:29:39.995809 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de218ae-1541-4d10-a48f-262aa27e1c7c service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:29:39.996079 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de218ae-1541-4d10-a48f-262aa27e1c7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:29:39.998053 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-9de218ae-1541-4d10-a48f-262aa27e1c7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:29:40.014672 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de218ae-1541-4d10-a48f-262aa27e1c7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:29:40.016373 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de218ae-1541-4d10-a48f-262aa27e1c7c service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:29:40.026043 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1401/2794] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:29:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:29:40.940964 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405b31db-b0c8-432d-8682-a6c84035f4e5 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:29:40.942705 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-405b31db-b0c8-432d-8682-a6c84035f4e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:29:41.021081 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405b31db-b0c8-432d-8682-a6c84035f4e5 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:29:41.026338 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ae8db3-9739-4793-8970-6f2923aa646d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71c2912a44f04bd983b4d4112dbee734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:29:41.026954 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae8db3-9739-4793-8970-6f2923aa646d service placement] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:29:41.027231 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae8db3-9739-4793-8970-6f2923aa646d service placement] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:29:41.027501 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae8db3-9739-4793-8970-6f2923aa646d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:29:41.029371 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-77ae8db3-9739-4793-8970-6f2923aa646d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:29:41.041788 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ae8db3-9739-4793-8970-6f2923aa646d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:29:41.043377 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ae8db3-9739-4793-8970-6f2923aa646d service placement] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:29:41.055531 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1394/2795] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:29:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:29:48.278152 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de218ae-1541-4d10-a48f-262aa27e1c7c service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:29:48.292328 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de218ae-1541-4d10-a48f-262aa27e1c7c service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:29:48.298450 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ee40d9-010a-4135-a986-c2ae73b421b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:29:48.299061 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ee40d9-010a-4135-a986-c2ae73b421b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:29:48.299338 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ee40d9-010a-4135-a986-c2ae73b421b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:29:48.299630 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ee40d9-010a-4135-a986-c2ae73b421b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:29:48.301805 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-90ee40d9-010a-4135-a986-c2ae73b421b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:29:48.371808 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ee40d9-010a-4135-a986-c2ae73b421b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:29:48.373333 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ee40d9-010a-4135-a986-c2ae73b421b4 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:29:48.383688 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1402/2796] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:29:48 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:29:51.050279 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=57 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:29:55.669340 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ae8db3-9739-4793-8970-6f2923aa646d service placement] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:29:55.670882 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-77ae8db3-9739-4793-8970-6f2923aa646d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:29:55.683777 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ae8db3-9739-4793-8970-6f2923aa646d service placement] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:29:55.687807 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc6c41f-bef4-45aa-9473-bace7837e85e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:29:55.688565 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc6c41f-bef4-45aa-9473-bace7837e85e service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:29:55.688886 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc6c41f-bef4-45aa-9473-bace7837e85e service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:29:55.689161 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc6c41f-bef4-45aa-9473-bace7837e85e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:29:55.691150 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ebc6c41f-bef4-45aa-9473-bace7837e85e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:29:55.760369 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc6c41f-bef4-45aa-9473-bace7837e85e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:29:55.761878 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc6c41f-bef4-45aa-9473-bace7837e85e service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:29:55.774019 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1395/2797] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:29:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:29:58.378651 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=34 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:29:58.890707 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ee40d9-010a-4135-a986-c2ae73b421b4 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:29:58.905308 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ee40d9-010a-4135-a986-c2ae73b421b4 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:29:58.910242 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154f4bf0-565e-4568-9924-e08b50391d88 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71c2912a44f04bd983b4d4112dbee734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:29:58.910861 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154f4bf0-565e-4568-9924-e08b50391d88 service placement] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:29:58.911178 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154f4bf0-565e-4568-9924-e08b50391d88 service placement] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:29:58.911444 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154f4bf0-565e-4568-9924-e08b50391d88 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:29:58.913403 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-154f4bf0-565e-4568-9924-e08b50391d88 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:29:58.928649 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154f4bf0-565e-4568-9924-e08b50391d88 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:29:58.930567 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154f4bf0-565e-4568-9924-e08b50391d88 service placement] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:29:58.941573 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1403/2798] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:29:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:30:02.573180 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc6c41f-bef4-45aa-9473-bace7837e85e service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:30:02.593211 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc6c41f-bef4-45aa-9473-bace7837e85e service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:30:02.599631 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e44422-476b-4dcd-b022-906ad2a5ecbb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71c2912a44f04bd983b4d4112dbee734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:30:02.600416 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e44422-476b-4dcd-b022-906ad2a5ecbb service placement] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:30:02.600827 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e44422-476b-4dcd-b022-906ad2a5ecbb service placement] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:30:02.601208 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e44422-476b-4dcd-b022-906ad2a5ecbb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:30:02.603146 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d0e44422-476b-4dcd-b022-906ad2a5ecbb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:30:02.617438 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e44422-476b-4dcd-b022-906ad2a5ecbb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:30:02.619290 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e44422-476b-4dcd-b022-906ad2a5ecbb service placement] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:30:02.636922 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1396/2799] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:30:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:30:02.649621 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154f4bf0-565e-4568-9924-e08b50391d88 service placement] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:30:02.651413 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-154f4bf0-565e-4568-9924-e08b50391d88 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:30:02.668728 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154f4bf0-565e-4568-9924-e08b50391d88 service placement] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:30:02.677919 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7b40f8-500c-4228-b5c4-aecea7d179c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:30:02.678853 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b40f8-500c-4228-b5c4-aecea7d179c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:30:02.679135 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b40f8-500c-4228-b5c4-aecea7d179c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:30:02.679589 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b40f8-500c-4228-b5c4-aecea7d179c2 admin admin] PATH_INFO: `/v3/users/e8bab2b0d23a47e19c4a50e72fc75714` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:30:02.681374 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7b40f8-500c-4228-b5c4-aecea7d179c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8bab2b0d23a47e19c4a50e72fc75714)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:30:02.683702 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7b40f8-500c-4228-b5c4-aecea7d179c2 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:30:02.739147 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-4c7b40f8-500c-4228-b5c4-aecea7d179c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8bab2b0d23a47e19c4a50e72fc75714'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:30:02.739832 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-4c7b40f8-500c-4228-b5c4-aecea7d179c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8bab2b0d23a47e19c4a50e72fc75714'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:30:02.747410 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-4c7b40f8-500c-4228-b5c4-aecea7d179c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8bab2b0d23a47e19c4a50e72fc75714'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:30:02.759826 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-4c7b40f8-500c-4228-b5c4-aecea7d179c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8bab2b0d23a47e19c4a50e72fc75714'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:30:02.760582 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-4c7b40f8-500c-4228-b5c4-aecea7d179c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8bab2b0d23a47e19c4a50e72fc75714'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:30:02.775734 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1404/2800] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:30:02 2022] DELETE /identity/v3/users/e8bab2b0d23a47e19c4a50e72fc75714 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:30:02.957655 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e44422-476b-4dcd-b022-906ad2a5ecbb service placement] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:30:02.970102 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e44422-476b-4dcd-b022-906ad2a5ecbb service placement] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:30:03.036610 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14cc586-5379-4a10-a7d4-740380e48c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:30:03.037306 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14cc586-5379-4a10-a7d4-740380e48c4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:30:03.037572 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14cc586-5379-4a10-a7d4-740380e48c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:30:03.037853 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14cc586-5379-4a10-a7d4-740380e48c4b admin admin] PATH_INFO: `/v3/projects/71b0e99f20e842c29ec46e5731ae9d9a` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:30:03.039078 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14cc586-5379-4a10-a7d4-740380e48c4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=71b0e99f20e842c29ec46e5731ae9d9a)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:30:03.041032 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14cc586-5379-4a10-a7d4-740380e48c4b admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:30:03.100593 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b14cc586-5379-4a10-a7d4-740380e48c4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71b0e99f20e842c29ec46e5731ae9d9a', 'request_id': 'req-b14cc586-5379-4a10-a7d4-740380e48c4b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:30:03.110450 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b14cc586-5379-4a10-a7d4-740380e48c4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71b0e99f20e842c29ec46e5731ae9d9a', 'request_id': 'req-b14cc586-5379-4a10-a7d4-740380e48c4b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:30:03.116704 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b14cc586-5379-4a10-a7d4-740380e48c4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71b0e99f20e842c29ec46e5731ae9d9a', 'request_id': 'req-b14cc586-5379-4a10-a7d4-740380e48c4b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:30:03.132550 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b14cc586-5379-4a10-a7d4-740380e48c4b admin admin] The token cache is being invalidate because project 71b0e99f20e842c29ec46e5731ae9d9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:30:03.132872 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b14cc586-5379-4a10-a7d4-740380e48c4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:30:03.133573 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b14cc586-5379-4a10-a7d4-740380e48c4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:30:03.135260 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1397/2801] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:30:02 2022] DELETE /identity/v3/projects/71b0e99f20e842c29ec46e5731ae9d9a => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:30:03.139711 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7b40f8-500c-4228-b5c4-aecea7d179c2 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:30:03.141408 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-4c7b40f8-500c-4228-b5c4-aecea7d179c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:30:03.155254 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7b40f8-500c-4228-b5c4-aecea7d179c2 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:30:03.223131 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36874bc-f639-4efc-8b48-0751c569c91a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:30:03.223878 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36874bc-f639-4efc-8b48-0751c569c91a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:30:03.224160 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36874bc-f639-4efc-8b48-0751c569c91a admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:30:03.224426 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36874bc-f639-4efc-8b48-0751c569c91a admin admin] PATH_INFO: `/v3/users/0289d3009cd549829062fdeb06ff39df` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:30:03.225619 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36874bc-f639-4efc-8b48-0751c569c91a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0289d3009cd549829062fdeb06ff39df)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:30:03.227543 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36874bc-f639-4efc-8b48-0751c569c91a admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:30:03.279496 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e36874bc-f639-4efc-8b48-0751c569c91a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0289d3009cd549829062fdeb06ff39df'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:30:03.280195 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e36874bc-f639-4efc-8b48-0751c569c91a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0289d3009cd549829062fdeb06ff39df'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:30:03.287098 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e36874bc-f639-4efc-8b48-0751c569c91a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0289d3009cd549829062fdeb06ff39df'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:30:03.292278 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:30:03.301336 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e36874bc-f639-4efc-8b48-0751c569c91a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0289d3009cd549829062fdeb06ff39df'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:30:03.302144 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e36874bc-f639-4efc-8b48-0751c569c91a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0289d3009cd549829062fdeb06ff39df'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:30:03.316055 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1405/2802] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:30:03 2022] DELETE /identity/v3/users/0289d3009cd549829062fdeb06ff39df => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:30:03.513848 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14cc586-5379-4a10-a7d4-740380e48c4b admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:30:03.515783 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b14cc586-5379-4a10-a7d4-740380e48c4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:30:03.531951 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14cc586-5379-4a10-a7d4-740380e48c4b admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:30:03.560437 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:30:03.621155 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30e1dd8-decf-4344-84e6-f1e4411139ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:30:03.622216 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30e1dd8-decf-4344-84e6-f1e4411139ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:30:03.622649 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30e1dd8-decf-4344-84e6-f1e4411139ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:30:03.623038 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30e1dd8-decf-4344-84e6-f1e4411139ca admin admin] PATH_INFO: `/v3/projects/b8ed1d1252d1476987a7a34dfaceb395` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:30:03.624453 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30e1dd8-decf-4344-84e6-f1e4411139ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8ed1d1252d1476987a7a34dfaceb395)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:30:03.626749 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30e1dd8-decf-4344-84e6-f1e4411139ca admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:30:03.685084 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c30e1dd8-decf-4344-84e6-f1e4411139ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8ed1d1252d1476987a7a34dfaceb395', 'request_id': 'req-c30e1dd8-decf-4344-84e6-f1e4411139ca'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:30:03.696031 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c30e1dd8-decf-4344-84e6-f1e4411139ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8ed1d1252d1476987a7a34dfaceb395', 'request_id': 'req-c30e1dd8-decf-4344-84e6-f1e4411139ca'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:30:03.704435 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c30e1dd8-decf-4344-84e6-f1e4411139ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8ed1d1252d1476987a7a34dfaceb395', 'request_id': 'req-c30e1dd8-decf-4344-84e6-f1e4411139ca'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:30:03.719831 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c30e1dd8-decf-4344-84e6-f1e4411139ca admin admin] The token cache is being invalidate because project b8ed1d1252d1476987a7a34dfaceb395 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:30:03.720275 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c30e1dd8-decf-4344-84e6-f1e4411139ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:30:03.721079 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c30e1dd8-decf-4344-84e6-f1e4411139ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:30:03.722731 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1398/2803] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:30:03 2022] DELETE /identity/v3/projects/b8ed1d1252d1476987a7a34dfaceb395 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:30:13.312497 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:DELETE=3,keystone:SELECT=6,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:30:13.716302 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:30:45.494181 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36874bc-f639-4efc-8b48-0751c569c91a admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:30:45.496194 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e36874bc-f639-4efc-8b48-0751c569c91a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:30:45.564868 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36874bc-f639-4efc-8b48-0751c569c91a admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:30:45.618171 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5c596a-6b92-46c9-849b-e691bc722c3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:30:45.618788 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c596a-6b92-46c9-849b-e691bc722c3e service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:30:45.619066 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c596a-6b92-46c9-849b-e691bc722c3e service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:30:45.619342 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c596a-6b92-46c9-849b-e691bc722c3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:30:45.621175 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-fc5c596a-6b92-46c9-849b-e691bc722c3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:30:45.698218 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5c596a-6b92-46c9-849b-e691bc722c3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:30:45.699681 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5c596a-6b92-46c9-849b-e691bc722c3e service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:30:45.711285 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1406/2804] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:30:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:30:51.224406 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30e1dd8-decf-4344-84e6-f1e4411139ca admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:30:51.226030 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c30e1dd8-decf-4344-84e6-f1e4411139ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:30:51.238860 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30e1dd8-decf-4344-84e6-f1e4411139ca admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:30:51.247961 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135ca4aa-6fbc-467a-935b-3ff339a9304f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:30:51.248975 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135ca4aa-6fbc-467a-935b-3ff339a9304f service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:30:51.249445 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135ca4aa-6fbc-467a-935b-3ff339a9304f service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:30:51.250182 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135ca4aa-6fbc-467a-935b-3ff339a9304f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:30:51.252337 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-135ca4aa-6fbc-467a-935b-3ff339a9304f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:30:51.267098 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135ca4aa-6fbc-467a-935b-3ff339a9304f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:30:51.268704 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135ca4aa-6fbc-467a-935b-3ff339a9304f service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:30:51.281392 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1399/2805] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:30:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:30:55.706347 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=74 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:30:57.045162 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5c596a-6b92-46c9-849b-e691bc722c3e service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:30:57.054961 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5c596a-6b92-46c9-849b-e691bc722c3e service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:30:57.058951 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b6254e-352e-4d62-a76b-bb0db2425ca7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:30:57.059581 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b6254e-352e-4d62-a76b-bb0db2425ca7 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:30:57.059850 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b6254e-352e-4d62-a76b-bb0db2425ca7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:30:57.060093 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b6254e-352e-4d62-a76b-bb0db2425ca7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:30:57.062016 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f9b6254e-352e-4d62-a76b-bb0db2425ca7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:30:57.128167 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b6254e-352e-4d62-a76b-bb0db2425ca7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:30:57.129652 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b6254e-352e-4d62-a76b-bb0db2425ca7 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:30:57.139701 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1407/2806] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:30:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:31:01.277605 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=6 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:31:03.703922 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135ca4aa-6fbc-467a-935b-3ff339a9304f service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:31:03.705629 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-135ca4aa-6fbc-467a-935b-3ff339a9304f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:31:03.721544 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135ca4aa-6fbc-467a-935b-3ff339a9304f service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:31:03.727239 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd81037-452b-423f-adee-47648810ab14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:31:03.727947 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd81037-452b-423f-adee-47648810ab14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:31:03.728146 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd81037-452b-423f-adee-47648810ab14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:31:03.728425 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd81037-452b-423f-adee-47648810ab14 admin admin] PATH_INFO: `/v3/users/4d3856b7bb7f4f5bb801952b893bbe35` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:31:03.729658 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd81037-452b-423f-adee-47648810ab14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d3856b7bb7f4f5bb801952b893bbe35)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:31:03.731891 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd81037-452b-423f-adee-47648810ab14 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:31:03.795908 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8fd81037-452b-423f-adee-47648810ab14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d3856b7bb7f4f5bb801952b893bbe35'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:31:03.797000 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8fd81037-452b-423f-adee-47648810ab14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d3856b7bb7f4f5bb801952b893bbe35'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:31:03.804452 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8fd81037-452b-423f-adee-47648810ab14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d3856b7bb7f4f5bb801952b893bbe35'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:31:03.819170 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8fd81037-452b-423f-adee-47648810ab14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d3856b7bb7f4f5bb801952b893bbe35'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:31:03.827790 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8fd81037-452b-423f-adee-47648810ab14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d3856b7bb7f4f5bb801952b893bbe35'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:31:03.834669 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1400/2807] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:31:03 2022] DELETE /identity/v3/users/4d3856b7bb7f4f5bb801952b893bbe35 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:31:04.022760 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b6254e-352e-4d62-a76b-bb0db2425ca7 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:31:04.037697 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b6254e-352e-4d62-a76b-bb0db2425ca7 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:31:04.103090 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b620c50d-627d-4ad2-9483-61a3370d3c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:31:04.103635 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b620c50d-627d-4ad2-9483-61a3370d3c4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:31:04.103911 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b620c50d-627d-4ad2-9483-61a3370d3c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:31:04.104182 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b620c50d-627d-4ad2-9483-61a3370d3c4d admin admin] PATH_INFO: `/v3/projects/f90ec559752b44b2b67c765488814865` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:31:04.105404 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b620c50d-627d-4ad2-9483-61a3370d3c4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f90ec559752b44b2b67c765488814865)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:31:04.107461 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b620c50d-627d-4ad2-9483-61a3370d3c4d admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:31:04.170555 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b620c50d-627d-4ad2-9483-61a3370d3c4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f90ec559752b44b2b67c765488814865', 'request_id': 'req-b620c50d-627d-4ad2-9483-61a3370d3c4d'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:31:04.176823 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b620c50d-627d-4ad2-9483-61a3370d3c4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f90ec559752b44b2b67c765488814865', 'request_id': 'req-b620c50d-627d-4ad2-9483-61a3370d3c4d'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:31:04.185956 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b620c50d-627d-4ad2-9483-61a3370d3c4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f90ec559752b44b2b67c765488814865', 'request_id': 'req-b620c50d-627d-4ad2-9483-61a3370d3c4d'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:31:04.199684 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b620c50d-627d-4ad2-9483-61a3370d3c4d admin admin] The token cache is being invalidate because project f90ec559752b44b2b67c765488814865 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:31:04.199982 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b620c50d-627d-4ad2-9483-61a3370d3c4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:31:04.200705 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b620c50d-627d-4ad2-9483-61a3370d3c4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:31:04.202269 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1408/2808] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:31:04 2022] DELETE /identity/v3/projects/f90ec559752b44b2b67c765488814865 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:31:04.206935 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd81037-452b-423f-adee-47648810ab14 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:31:04.208483 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8fd81037-452b-423f-adee-47648810ab14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:31:04.222927 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd81037-452b-423f-adee-47648810ab14 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:31:04.295786 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba20ef1-33a6-478b-8bc9-313f948039f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:31:04.296789 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba20ef1-33a6-478b-8bc9-313f948039f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:31:04.297098 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba20ef1-33a6-478b-8bc9-313f948039f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:31:04.297373 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba20ef1-33a6-478b-8bc9-313f948039f1 admin admin] PATH_INFO: `/v3/users/002b7089d7424a73ad28612a6007c471` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:31:04.298630 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba20ef1-33a6-478b-8bc9-313f948039f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=002b7089d7424a73ad28612a6007c471)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:31:04.300581 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba20ef1-33a6-478b-8bc9-313f948039f1 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:31:04.358822 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-bba20ef1-33a6-478b-8bc9-313f948039f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '002b7089d7424a73ad28612a6007c471'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:31:04.359724 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-bba20ef1-33a6-478b-8bc9-313f948039f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '002b7089d7424a73ad28612a6007c471'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:31:04.368102 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-bba20ef1-33a6-478b-8bc9-313f948039f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '002b7089d7424a73ad28612a6007c471'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:31:04.381300 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-bba20ef1-33a6-478b-8bc9-313f948039f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '002b7089d7424a73ad28612a6007c471'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:31:04.383823 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:31:04.394597 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-bba20ef1-33a6-478b-8bc9-313f948039f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '002b7089d7424a73ad28612a6007c471'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:31:04.402198 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1401/2809] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:31:04 2022] DELETE /identity/v3/users/002b7089d7424a73ad28612a6007c471 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:31:04.597485 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b620c50d-627d-4ad2-9483-61a3370d3c4d admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:31:04.601128 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b620c50d-627d-4ad2-9483-61a3370d3c4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:31:04.618050 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b620c50d-627d-4ad2-9483-61a3370d3c4d admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:31:04.642358 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:31:04.684172 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204bb247-60ca-4faa-b374-c8b0a083185b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:31:04.684851 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204bb247-60ca-4faa-b374-c8b0a083185b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:31:04.685130 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204bb247-60ca-4faa-b374-c8b0a083185b admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:31:04.685399 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204bb247-60ca-4faa-b374-c8b0a083185b admin admin] PATH_INFO: `/v3/projects/8629993381d04142aea2ca6ae0a49690` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:31:04.686515 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204bb247-60ca-4faa-b374-c8b0a083185b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8629993381d04142aea2ca6ae0a49690)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:31:04.688454 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204bb247-60ca-4faa-b374-c8b0a083185b admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:31:04.747639 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-204bb247-60ca-4faa-b374-c8b0a083185b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8629993381d04142aea2ca6ae0a49690', 'request_id': 'req-204bb247-60ca-4faa-b374-c8b0a083185b'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:31:04.758103 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-204bb247-60ca-4faa-b374-c8b0a083185b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8629993381d04142aea2ca6ae0a49690', 'request_id': 'req-204bb247-60ca-4faa-b374-c8b0a083185b'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:31:04.768905 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-204bb247-60ca-4faa-b374-c8b0a083185b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8629993381d04142aea2ca6ae0a49690', 'request_id': 'req-204bb247-60ca-4faa-b374-c8b0a083185b'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:31:04.783012 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-204bb247-60ca-4faa-b374-c8b0a083185b admin admin] The token cache is being invalidate because project 8629993381d04142aea2ca6ae0a49690 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:31:04.783292 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-204bb247-60ca-4faa-b374-c8b0a083185b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:31:04.784026 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-204bb247-60ca-4faa-b374-c8b0a083185b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:31:04.785689 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1409/2810] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:31:04 2022] DELETE /identity/v3/projects/8629993381d04142aea2ca6ae0a49690 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:31:14.399170 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:31:14.779302 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:31:35.508404 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28d4e0b-2594-446e-b5b1-8e96cd341804 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:31:35.509107 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28d4e0b-2594-446e-b5b1-8e96cd341804 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:31:35.509107 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28d4e0b-2594-446e-b5b1-8e96cd341804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:31:35.529995 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-b28d4e0b-2594-446e-b5b1-8e96cd341804 None None] MFA Rules not processed for user `523faa1ac64e4044b9fb651ec28249d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:31:35.531290 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b28d4e0b-2594-446e-b5b1-8e96cd341804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:31:35.595987 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1402/2811] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:31:35 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:31:45.588887 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=25 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:33:04.904413 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204bb247-60ca-4faa-b374-c8b0a083185b admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:04.906888 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-204bb247-60ca-4faa-b374-c8b0a083185b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:33:04.988477 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204bb247-60ca-4faa-b374-c8b0a083185b admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:04.995067 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9e01a4-b2a8-4d7d-9c4a-f57806cb3c1f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63de41f4348243998029f7d07aa5dc40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:04.995820 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9e01a4-b2a8-4d7d-9c4a-f57806cb3c1f service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:04.996158 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9e01a4-b2a8-4d7d-9c4a-f57806cb3c1f service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:04.996494 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9e01a4-b2a8-4d7d-9c4a-f57806cb3c1f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:04.998436 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0a9e01a4-b2a8-4d7d-9c4a-f57806cb3c1f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:33:05.009586 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9e01a4-b2a8-4d7d-9c4a-f57806cb3c1f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:05.011282 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9e01a4-b2a8-4d7d-9c4a-f57806cb3c1f service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:05.020786 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1410/2812] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:33:04 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:33:06.156918 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28d4e0b-2594-446e-b5b1-8e96cd341804 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:06.158460 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b28d4e0b-2594-446e-b5b1-8e96cd341804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:33:06.230507 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28d4e0b-2594-446e-b5b1-8e96cd341804 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:06.234294 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af51b17-652c-4c69-9b27-6f9ea3591199 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:06.234895 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af51b17-652c-4c69-9b27-6f9ea3591199 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:06.235177 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af51b17-652c-4c69-9b27-6f9ea3591199 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:06.235450 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af51b17-652c-4c69-9b27-6f9ea3591199 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:06.237374 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-2af51b17-652c-4c69-9b27-6f9ea3591199 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:33:06.247893 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af51b17-652c-4c69-9b27-6f9ea3591199 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:06.249345 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af51b17-652c-4c69-9b27-6f9ea3591199 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:06.258213 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1403/2813] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:33:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:33:15.016753 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=34 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:33:16.254090 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=26 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:33:18.872894 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9e01a4-b2a8-4d7d-9c4a-f57806cb3c1f service nova] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:18.886095 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9e01a4-b2a8-4d7d-9c4a-f57806cb3c1f service nova] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:18.892958 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96c56b9-f232-486b-80f8-ba233562a6c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63de41f4348243998029f7d07aa5dc40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:18.893632 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96c56b9-f232-486b-80f8-ba233562a6c0 service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:18.893912 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96c56b9-f232-486b-80f8-ba233562a6c0 service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:18.894180 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96c56b9-f232-486b-80f8-ba233562a6c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:18.896374 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b96c56b9-f232-486b-80f8-ba233562a6c0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:33:18.910754 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96c56b9-f232-486b-80f8-ba233562a6c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:18.912559 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96c56b9-f232-486b-80f8-ba233562a6c0 service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:18.924520 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1411/2814] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:33:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:33:21.502303 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5522de-cb50-417d-8d97-a885778a7b7b None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:21.502963 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5522de-cb50-417d-8d97-a885778a7b7b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:21.502963 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5522de-cb50-417d-8d97-a885778a7b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:21.523927 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-8d5522de-cb50-417d-8d97-a885778a7b7b None None] MFA Rules not processed for user `523faa1ac64e4044b9fb651ec28249d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:33:21.525198 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8d5522de-cb50-417d-8d97-a885778a7b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:33:21.533542 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1404/2815] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:33:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:33:22.192472 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96c56b9-f232-486b-80f8-ba233562a6c0 service nova] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:22.193962 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b96c56b9-f232-486b-80f8-ba233562a6c0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:33:22.281905 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96c56b9-f232-486b-80f8-ba233562a6c0 service nova] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:22.287178 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f007c4-a0a6-4a3a-b374-e2ee7cbee497 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71c2912a44f04bd983b4d4112dbee734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:22.287785 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f007c4-a0a6-4a3a-b374-e2ee7cbee497 service placement] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:22.288075 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f007c4-a0a6-4a3a-b374-e2ee7cbee497 service placement] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:22.288344 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f007c4-a0a6-4a3a-b374-e2ee7cbee497 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:22.290164 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-47f007c4-a0a6-4a3a-b374-e2ee7cbee497 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:33:22.303899 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f007c4-a0a6-4a3a-b374-e2ee7cbee497 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:22.305474 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f007c4-a0a6-4a3a-b374-e2ee7cbee497 service placement] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:22.316800 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1412/2816] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:33:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:33:30.361736 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5522de-cb50-417d-8d97-a885778a7b7b None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:30.378751 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5522de-cb50-417d-8d97-a885778a7b7b None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:30.385129 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2225ab-395a-47dd-8887-edc5e7649955 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63de41f4348243998029f7d07aa5dc40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:30.386114 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2225ab-395a-47dd-8887-edc5e7649955 service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:30.386591 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2225ab-395a-47dd-8887-edc5e7649955 service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:30.387073 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2225ab-395a-47dd-8887-edc5e7649955 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:30.389950 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-6d2225ab-395a-47dd-8887-edc5e7649955 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:33:30.409301 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2225ab-395a-47dd-8887-edc5e7649955 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:30.411172 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2225ab-395a-47dd-8887-edc5e7649955 service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:30.425748 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1405/2817] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:33:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:33:32.311573 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=38 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:33:40.426038 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=9 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:34:52.873847 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f007c4-a0a6-4a3a-b374-e2ee7cbee497 service placement] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:34:52.875346 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-47f007c4-a0a6-4a3a-b374-e2ee7cbee497 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:34:52.969814 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f007c4-a0a6-4a3a-b374-e2ee7cbee497 service placement] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:34:52.975120 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d66283-ae9a-4ca5-9eb4-667643c8b6ef service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71c2912a44f04bd983b4d4112dbee734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:34:52.975740 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d66283-ae9a-4ca5-9eb4-667643c8b6ef service placement] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:34:52.976017 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d66283-ae9a-4ca5-9eb4-667643c8b6ef service placement] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:34:52.976290 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d66283-ae9a-4ca5-9eb4-667643c8b6ef service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:34:52.978071 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a0d66283-ae9a-4ca5-9eb4-667643c8b6ef service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:34:52.990535 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d66283-ae9a-4ca5-9eb4-667643c8b6ef service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:34:52.992198 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d66283-ae9a-4ca5-9eb4-667643c8b6ef service placement] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:34:53.002503 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1413/2818] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:34:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:34:58.651946 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2225ab-395a-47dd-8887-edc5e7649955 service nova] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:34:58.653747 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-6d2225ab-395a-47dd-8887-edc5e7649955 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:34:58.726958 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2225ab-395a-47dd-8887-edc5e7649955 service nova] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:34:58.730947 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950f63b8-85ba-4963-8523-fae59acd5447 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:34:58.731561 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950f63b8-85ba-4963-8523-fae59acd5447 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:34:58.731835 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950f63b8-85ba-4963-8523-fae59acd5447 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:34:58.732104 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950f63b8-85ba-4963-8523-fae59acd5447 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:34:58.733899 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-950f63b8-85ba-4963-8523-fae59acd5447 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:34:58.823582 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950f63b8-85ba-4963-8523-fae59acd5447 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:34:58.825098 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950f63b8-85ba-4963-8523-fae59acd5447 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:34:58.843877 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1406/2819] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:34:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:35:01.892116 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d66283-ae9a-4ca5-9eb4-667643c8b6ef service placement] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:01.904359 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d66283-ae9a-4ca5-9eb4-667643c8b6ef service placement] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:01.909627 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d67aa16-0824-41f3-af65-a7c12745deda service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71c2912a44f04bd983b4d4112dbee734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:01.910244 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d67aa16-0824-41f3-af65-a7c12745deda service placement] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:01.910513 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d67aa16-0824-41f3-af65-a7c12745deda service placement] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:01.910974 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d67aa16-0824-41f3-af65-a7c12745deda service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:01.913151 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5d67aa16-0824-41f3-af65-a7c12745deda service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:35:01.934972 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d67aa16-0824-41f3-af65-a7c12745deda service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:01.936691 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d67aa16-0824-41f3-af65-a7c12745deda service placement] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:01.949866 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950f63b8-85ba-4963-8523-fae59acd5447 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:01.951483 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-950f63b8-85ba-4963-8523-fae59acd5447 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:35:01.954018 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1414/2820] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:35:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:35:01.970266 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950f63b8-85ba-4963-8523-fae59acd5447 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:01.976519 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52c459a-cde7-48cb-8981-d0c987d16708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:01.977191 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c459a-cde7-48cb-8981-d0c987d16708 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:01.977529 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c459a-cde7-48cb-8981-d0c987d16708 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:01.977864 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c459a-cde7-48cb-8981-d0c987d16708 admin admin] PATH_INFO: `/v3/users/523faa1ac64e4044b9fb651ec28249d9` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:01.979065 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52c459a-cde7-48cb-8981-d0c987d16708 admin admin] RBAC: Authorizing `identity:delete_user(user_id=523faa1ac64e4044b9fb651ec28249d9)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:01.981150 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52c459a-cde7-48cb-8981-d0c987d16708 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:02.028295 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-f52c459a-cde7-48cb-8981-d0c987d16708 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '523faa1ac64e4044b9fb651ec28249d9'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:02.028976 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-f52c459a-cde7-48cb-8981-d0c987d16708 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '523faa1ac64e4044b9fb651ec28249d9'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:02.035775 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-f52c459a-cde7-48cb-8981-d0c987d16708 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '523faa1ac64e4044b9fb651ec28249d9'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:02.047296 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:35:02.048452 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-f52c459a-cde7-48cb-8981-d0c987d16708 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '523faa1ac64e4044b9fb651ec28249d9'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:02.057096 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-f52c459a-cde7-48cb-8981-d0c987d16708 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '523faa1ac64e4044b9fb651ec28249d9'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:02.063084 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1407/2821] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:35:01 2022] DELETE /identity/v3/users/523faa1ac64e4044b9fb651ec28249d9 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:35:02.249065 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d67aa16-0824-41f3-af65-a7c12745deda service placement] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:02.260644 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d67aa16-0824-41f3-af65-a7c12745deda service placement] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:02.326989 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2654a52-c86e-4b90-842a-cb6185585ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:02.327735 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2654a52-c86e-4b90-842a-cb6185585ea0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:02.328012 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2654a52-c86e-4b90-842a-cb6185585ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:02.328282 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2654a52-c86e-4b90-842a-cb6185585ea0 admin admin] PATH_INFO: `/v3/projects/5c722f58536f492aa4eb96085452def4` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:02.329408 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2654a52-c86e-4b90-842a-cb6185585ea0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c722f58536f492aa4eb96085452def4)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:02.331329 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2654a52-c86e-4b90-842a-cb6185585ea0 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:02.379430 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-c2654a52-c86e-4b90-842a-cb6185585ea0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c722f58536f492aa4eb96085452def4', 'request_id': 'req-c2654a52-c86e-4b90-842a-cb6185585ea0'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:02.384907 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-c2654a52-c86e-4b90-842a-cb6185585ea0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c722f58536f492aa4eb96085452def4', 'request_id': 'req-c2654a52-c86e-4b90-842a-cb6185585ea0'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:02.393145 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-c2654a52-c86e-4b90-842a-cb6185585ea0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c722f58536f492aa4eb96085452def4', 'request_id': 'req-c2654a52-c86e-4b90-842a-cb6185585ea0'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:02.395146 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:35:02.406327 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-c2654a52-c86e-4b90-842a-cb6185585ea0 admin admin] The token cache is being invalidate because project 5c722f58536f492aa4eb96085452def4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:35:02.406877 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-c2654a52-c86e-4b90-842a-cb6185585ea0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:02.407981 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-c2654a52-c86e-4b90-842a-cb6185585ea0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:02.409607 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1415/2822] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:35:02 2022] DELETE /identity/v3/projects/5c722f58536f492aa4eb96085452def4 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:35:02.414551 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd75c689-0010-4663-9f24-d38d69d95802 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:02.414838 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd75c689-0010-4663-9f24-d38d69d95802 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:02.415110 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd75c689-0010-4663-9f24-d38d69d95802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:02.481544 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-cd75c689-0010-4663-9f24-d38d69d95802 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:35:02.482669 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-cd75c689-0010-4663-9f24-d38d69d95802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:35:02.542282 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1408/2823] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:35:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:35:02.550715 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb98974d-de8e-43f7-b29c-94e6823934e5 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:02.551042 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb98974d-de8e-43f7-b29c-94e6823934e5 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:02.551368 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb98974d-de8e-43f7-b29c-94e6823934e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:02.573461 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-eb98974d-de8e-43f7-b29c-94e6823934e5 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:35:02.574311 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-eb98974d-de8e-43f7-b29c-94e6823934e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:35:02.583493 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1416/2824] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:35:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:35:02.588207 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd75c689-0010-4663-9f24-d38d69d95802 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:02.598486 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd75c689-0010-4663-9f24-d38d69d95802 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:02.603749 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226fadd4-a1d6-480b-90df-c76fd7d4ae6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:02.604455 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226fadd4-a1d6-480b-90df-c76fd7d4ae6a admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:02.604869 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226fadd4-a1d6-480b-90df-c76fd7d4ae6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:02.605200 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226fadd4-a1d6-480b-90df-c76fd7d4ae6a admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:02.605920 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226fadd4-a1d6-480b-90df-c76fd7d4ae6a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:35:02.606282 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226fadd4-a1d6-480b-90df-c76fd7d4ae6a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:02.608066 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226fadd4-a1d6-480b-90df-c76fd7d4ae6a admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:02.625700 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1409/2825] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 6 16:35:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:35:02.629455 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb98974d-de8e-43f7-b29c-94e6823934e5 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:02.639516 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb98974d-de8e-43f7-b29c-94e6823934e5 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:02.645204 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c3c2b9-1d56-4d44-9efc-fe48755ca60c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:02.645815 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c3c2b9-1d56-4d44-9efc-fe48755ca60c admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:02.646092 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c3c2b9-1d56-4d44-9efc-fe48755ca60c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:02.646357 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c3c2b9-1d56-4d44-9efc-fe48755ca60c admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:02.646986 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c3c2b9-1d56-4d44-9efc-fe48755ca60c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:02.648984 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c3c2b9-1d56-4d44-9efc-fe48755ca60c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:02.683523 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1417/2826] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:35:02 2022] POST /identity/v3/projects => generated 363 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:35:02.684892 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226fadd4-a1d6-480b-90df-c76fd7d4ae6a admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:02.695859 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226fadd4-a1d6-480b-90df-c76fd7d4ae6a admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:02.754678 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cd0d28-44d8-4c8c-af13-e946f03eff35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:02.755285 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cd0d28-44d8-4c8c-af13-e946f03eff35 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:02.755556 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cd0d28-44d8-4c8c-af13-e946f03eff35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:02.755826 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cd0d28-44d8-4c8c-af13-e946f03eff35 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:02.756450 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cd0d28-44d8-4c8c-af13-e946f03eff35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:02.758514 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cd0d28-44d8-4c8c-af13-e946f03eff35 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:02.781257 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1410/2827] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:35:02 2022] POST /identity/v3/users => generated 282 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:35:02.785590 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c3c2b9-1d56-4d44-9efc-fe48755ca60c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:02.796948 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c3c2b9-1d56-4d44-9efc-fe48755ca60c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:02.802959 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd8543f-1b7b-4ae5-ba5c-d202fd6bd05d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:02.803596 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd8543f-1b7b-4ae5-ba5c-d202fd6bd05d admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:02.803873 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd8543f-1b7b-4ae5-ba5c-d202fd6bd05d admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:02.804147 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd8543f-1b7b-4ae5-ba5c-d202fd6bd05d admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:02.804817 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd8543f-1b7b-4ae5-ba5c-d202fd6bd05d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:02.806447 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd8543f-1b7b-4ae5-ba5c-d202fd6bd05d admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:02.817078 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1418/2828] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:35:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:35:02.821322 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cd0d28-44d8-4c8c-af13-e946f03eff35 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:02.832571 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cd0d28-44d8-4c8c-af13-e946f03eff35 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:02.838847 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2d457b-a731-4a9a-bf47-b84af9b0292b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:02.840371 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2d457b-a731-4a9a-bf47-b84af9b0292b admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:02.840669 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2d457b-a731-4a9a-bf47-b84af9b0292b admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:02.840974 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2d457b-a731-4a9a-bf47-b84af9b0292b admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:02.841657 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2d457b-a731-4a9a-bf47-b84af9b0292b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:02.843398 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2d457b-a731-4a9a-bf47-b84af9b0292b admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:02.855062 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1411/2829] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:35:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:35:02.859488 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd8543f-1b7b-4ae5-ba5c-d202fd6bd05d admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:02.870151 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd8543f-1b7b-4ae5-ba5c-d202fd6bd05d admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:02.875679 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44583de-813e-43fd-ab77-2b44be4ea747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:02.876337 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44583de-813e-43fd-ab77-2b44be4ea747 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:02.876616 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44583de-813e-43fd-ab77-2b44be4ea747 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:02.876925 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44583de-813e-43fd-ab77-2b44be4ea747 admin admin] PATH_INFO: `/v3/projects/0ad17446f71e45578bd0c6fed5fefa8d/users/17afae5ab1ad478685542656ec9ba275/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:02.892686 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44583de-813e-43fd-ab77-2b44be4ea747 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ad17446f71e45578bd0c6fed5fefa8d, user_id=17afae5ab1ad478685542656ec9ba275, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:02.895423 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44583de-813e-43fd-ab77-2b44be4ea747 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:02.908591 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1419/2830] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:35:02 2022] PUT /identity/v3/projects/0ad17446f71e45578bd0c6fed5fefa8d/users/17afae5ab1ad478685542656ec9ba275/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:35:02.919854 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2643e5-aec6-40f2-a55f-c3a7582855e5 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:02.920174 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2643e5-aec6-40f2-a55f-c3a7582855e5 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:02.920483 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2643e5-aec6-40f2-a55f-c3a7582855e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:02.948822 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-bd2643e5-aec6-40f2-a55f-c3a7582855e5 None None] MFA Rules not processed for user `17afae5ab1ad478685542656ec9ba275`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:35:02.950022 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-bd2643e5-aec6-40f2-a55f-c3a7582855e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:35:02.962322 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:35:03.017898 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1412/2831] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:35:02 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:35:03.022632 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44583de-813e-43fd-ab77-2b44be4ea747 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:03.084833 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44583de-813e-43fd-ab77-2b44be4ea747 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:03.090015 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58968ac2-4d31-4b2b-8156-5cc03fc50471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:03.090617 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58968ac2-4d31-4b2b-8156-5cc03fc50471 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:03.090901 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58968ac2-4d31-4b2b-8156-5cc03fc50471 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:03.091170 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58968ac2-4d31-4b2b-8156-5cc03fc50471 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:03.091799 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58968ac2-4d31-4b2b-8156-5cc03fc50471 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:03.093746 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58968ac2-4d31-4b2b-8156-5cc03fc50471 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:03.122884 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1420/2832] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:35:03 2022] POST /identity/v3/projects => generated 365 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:35:03.127142 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2643e5-aec6-40f2-a55f-c3a7582855e5 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:03.137901 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2643e5-aec6-40f2-a55f-c3a7582855e5 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:03.195015 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d33088-fb10-44eb-baad-433f626f21c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:03.195618 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d33088-fb10-44eb-baad-433f626f21c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:03.195894 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d33088-fb10-44eb-baad-433f626f21c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:03.196161 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d33088-fb10-44eb-baad-433f626f21c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:03.196801 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d33088-fb10-44eb-baad-433f626f21c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:03.198735 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d33088-fb10-44eb-baad-433f626f21c4 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:03.220482 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1413/2833] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:35:03 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:35:03.224475 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58968ac2-4d31-4b2b-8156-5cc03fc50471 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:03.235133 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58968ac2-4d31-4b2b-8156-5cc03fc50471 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:03.240239 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c727c5ae-9e01-4f27-991e-4517e2e10b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:03.240903 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c727c5ae-9e01-4f27-991e-4517e2e10b45 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:03.241181 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c727c5ae-9e01-4f27-991e-4517e2e10b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:03.241454 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c727c5ae-9e01-4f27-991e-4517e2e10b45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:03.242116 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c727c5ae-9e01-4f27-991e-4517e2e10b45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:03.243785 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c727c5ae-9e01-4f27-991e-4517e2e10b45 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:03.255482 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1421/2834] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:35:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:35:03.259953 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d33088-fb10-44eb-baad-433f626f21c4 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:03.271148 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d33088-fb10-44eb-baad-433f626f21c4 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:03.277255 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818185db-0e90-459f-8824-64f3df72e8c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:03.277936 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818185db-0e90-459f-8824-64f3df72e8c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:03.278217 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818185db-0e90-459f-8824-64f3df72e8c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:03.278602 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818185db-0e90-459f-8824-64f3df72e8c8 admin admin] PATH_INFO: `/v3/projects/6d168e52e832405296b0692ae8df6279/users/5533dca4f7554684a6ef3d70ecf16b2c/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:03.294586 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818185db-0e90-459f-8824-64f3df72e8c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d168e52e832405296b0692ae8df6279, user_id=5533dca4f7554684a6ef3d70ecf16b2c, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:03.297278 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818185db-0e90-459f-8824-64f3df72e8c8 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:03.311537 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1414/2835] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:35:03 2022] PUT /identity/v3/projects/6d168e52e832405296b0692ae8df6279/users/5533dca4f7554684a6ef3d70ecf16b2c/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:35:03.316736 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c727c5ae-9e01-4f27-991e-4517e2e10b45 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:03.327174 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c727c5ae-9e01-4f27-991e-4517e2e10b45 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:03.391175 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0515273-bc69-4178-9616-d662f2ac9145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:03.391792 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0515273-bc69-4178-9616-d662f2ac9145 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:03.392071 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0515273-bc69-4178-9616-d662f2ac9145 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:03.392332 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0515273-bc69-4178-9616-d662f2ac9145 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:03.393006 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0515273-bc69-4178-9616-d662f2ac9145 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:03.394606 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0515273-bc69-4178-9616-d662f2ac9145 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:03.404825 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1422/2836] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:35:03 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:35:03.409324 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818185db-0e90-459f-8824-64f3df72e8c8 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:03.420793 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818185db-0e90-459f-8824-64f3df72e8c8 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:03.426065 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8389c184-a833-4ff1-943f-d7a038c9cc1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:03.426739 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8389c184-a833-4ff1-943f-d7a038c9cc1d admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:03.427017 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8389c184-a833-4ff1-943f-d7a038c9cc1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:03.427290 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8389c184-a833-4ff1-943f-d7a038c9cc1d admin admin] PATH_INFO: `/v3/projects/6d168e52e832405296b0692ae8df6279/users/5533dca4f7554684a6ef3d70ecf16b2c/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:03.429629 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8389c184-a833-4ff1-943f-d7a038c9cc1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d168e52e832405296b0692ae8df6279, user_id=5533dca4f7554684a6ef3d70ecf16b2c, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:03.432323 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8389c184-a833-4ff1-943f-d7a038c9cc1d admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:03.443173 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1415/2837] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:35:03 2022] PUT /identity/v3/projects/6d168e52e832405296b0692ae8df6279/users/5533dca4f7554684a6ef3d70ecf16b2c/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:35:03.454968 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc4b5f0-b40b-4ce7-bf85-812278d0171d None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:03.455211 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc4b5f0-b40b-4ce7-bf85-812278d0171d None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:03.455504 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc4b5f0-b40b-4ce7-bf85-812278d0171d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:03.462718 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:35:03.489620 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-5cc4b5f0-b40b-4ce7-bf85-812278d0171d None None] MFA Rules not processed for user `5533dca4f7554684a6ef3d70ecf16b2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:35:03.490846 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5cc4b5f0-b40b-4ce7-bf85-812278d0171d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:35:03.560869 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1423/2838] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:35:03 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:35:03.573061 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8389c184-a833-4ff1-943f-d7a038c9cc1d admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:03.575000 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8389c184-a833-4ff1-943f-d7a038c9cc1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:35:03.639214 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8389c184-a833-4ff1-943f-d7a038c9cc1d admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:03.643410 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7109d4-d857-408b-a3e1-53cbb047e378 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:03.644057 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7109d4-d857-408b-a3e1-53cbb047e378 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:03.644353 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7109d4-d857-408b-a3e1-53cbb047e378 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:03.644620 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7109d4-d857-408b-a3e1-53cbb047e378 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:03.654967 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7109d4-d857-408b-a3e1-53cbb047e378 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:03.656414 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7109d4-d857-408b-a3e1-53cbb047e378 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:03.666379 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1416/2839] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:35:03 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:35:03.746668 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc4b5f0-b40b-4ce7-bf85-812278d0171d None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:03.757449 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc4b5f0-b40b-4ce7-bf85-812278d0171d None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:03.762053 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67af2092-1e55-474b-88d3-720362db734a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:03.762695 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af2092-1e55-474b-88d3-720362db734a service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:03.762970 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af2092-1e55-474b-88d3-720362db734a service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:03.763237 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af2092-1e55-474b-88d3-720362db734a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:03.821737 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67af2092-1e55-474b-88d3-720362db734a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:03.823106 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67af2092-1e55-474b-88d3-720362db734a service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:03.831899 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1424/2840] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:35:03 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:35:06.599698 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7109d4-d857-408b-a3e1-53cbb047e378 service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:06.609998 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:35:06.664919 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7109d4-d857-408b-a3e1-53cbb047e378 service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:06.670326 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5f8c85-f321-4462-850a-6d675f966193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:06.670955 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5f8c85-f321-4462-850a-6d675f966193 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:06.671232 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5f8c85-f321-4462-850a-6d675f966193 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:06.671506 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5f8c85-f321-4462-850a-6d675f966193 admin admin] PATH_INFO: `/v3/users/17afae5ab1ad478685542656ec9ba275` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:06.672674 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5f8c85-f321-4462-850a-6d675f966193 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17afae5ab1ad478685542656ec9ba275)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:06.674640 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5f8c85-f321-4462-850a-6d675f966193 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:06.723527 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1e5f8c85-f321-4462-850a-6d675f966193 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17afae5ab1ad478685542656ec9ba275'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:06.724268 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1e5f8c85-f321-4462-850a-6d675f966193 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17afae5ab1ad478685542656ec9ba275'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:06.732309 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1e5f8c85-f321-4462-850a-6d675f966193 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17afae5ab1ad478685542656ec9ba275'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:06.745441 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1e5f8c85-f321-4462-850a-6d675f966193 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17afae5ab1ad478685542656ec9ba275'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:06.754161 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1e5f8c85-f321-4462-850a-6d675f966193 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17afae5ab1ad478685542656ec9ba275'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:06.760725 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1417/2841] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:35:06 2022] DELETE /identity/v3/users/17afae5ab1ad478685542656ec9ba275 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:35:06.765440 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86492ab-aa77-4623-81ba-c27ff1ba7045 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:06.765976 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86492ab-aa77-4623-81ba-c27ff1ba7045 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:06.765976 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86492ab-aa77-4623-81ba-c27ff1ba7045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:06.785206 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-c86492ab-aa77-4623-81ba-c27ff1ba7045 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:35:06.786425 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-c86492ab-aa77-4623-81ba-c27ff1ba7045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:35:06.847686 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1425/2842] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:35:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:35:06.857563 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5f8c85-f321-4462-850a-6d675f966193 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:06.859179 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-1e5f8c85-f321-4462-850a-6d675f966193 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:35:06.920968 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5f8c85-f321-4462-850a-6d675f966193 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:06.924840 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc57ec30-ced6-4a6a-8a80-54108af37f02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:06.925447 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc57ec30-ced6-4a6a-8a80-54108af37f02 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:06.925719 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc57ec30-ced6-4a6a-8a80-54108af37f02 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:06.925983 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc57ec30-ced6-4a6a-8a80-54108af37f02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:06.935091 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc57ec30-ced6-4a6a-8a80-54108af37f02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:06.936543 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc57ec30-ced6-4a6a-8a80-54108af37f02 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:06.945909 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1418/2843] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:35:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:35:07.216674 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86492ab-aa77-4623-81ba-c27ff1ba7045 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:07.218181 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-c86492ab-aa77-4623-81ba-c27ff1ba7045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:35:07.231156 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86492ab-aa77-4623-81ba-c27ff1ba7045 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:07.236432 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58de410b-e50e-4c71-994d-5f33d9db8d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:07.237109 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58de410b-e50e-4c71-994d-5f33d9db8d1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:07.237389 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58de410b-e50e-4c71-994d-5f33d9db8d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:07.237663 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58de410b-e50e-4c71-994d-5f33d9db8d1b admin admin] PATH_INFO: `/v3/projects/0ad17446f71e45578bd0c6fed5fefa8d` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:07.238872 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58de410b-e50e-4c71-994d-5f33d9db8d1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ad17446f71e45578bd0c6fed5fefa8d)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:07.240930 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58de410b-e50e-4c71-994d-5f33d9db8d1b admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:07.265819 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:35:07.293056 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-58de410b-e50e-4c71-994d-5f33d9db8d1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ad17446f71e45578bd0c6fed5fefa8d', 'request_id': 'req-58de410b-e50e-4c71-994d-5f33d9db8d1b'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:07.299366 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-58de410b-e50e-4c71-994d-5f33d9db8d1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ad17446f71e45578bd0c6fed5fefa8d', 'request_id': 'req-58de410b-e50e-4c71-994d-5f33d9db8d1b'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:07.308479 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-58de410b-e50e-4c71-994d-5f33d9db8d1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ad17446f71e45578bd0c6fed5fefa8d', 'request_id': 'req-58de410b-e50e-4c71-994d-5f33d9db8d1b'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:07.320100 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-58de410b-e50e-4c71-994d-5f33d9db8d1b admin admin] The token cache is being invalidate because project 0ad17446f71e45578bd0c6fed5fefa8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:35:07.320390 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-58de410b-e50e-4c71-994d-5f33d9db8d1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:07.321084 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-58de410b-e50e-4c71-994d-5f33d9db8d1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:07.322669 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1426/2844] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:35:07 2022] DELETE /identity/v3/projects/0ad17446f71e45578bd0c6fed5fefa8d => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:35:07.326932 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc57ec30-ced6-4a6a-8a80-54108af37f02 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:07.337111 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc57ec30-ced6-4a6a-8a80-54108af37f02 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:07.368023 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:35:07.397953 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e19c25e-cad4-4a9c-9867-b69846a1f4b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:07.398588 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e19c25e-cad4-4a9c-9867-b69846a1f4b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:07.398865 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e19c25e-cad4-4a9c-9867-b69846a1f4b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:07.399137 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e19c25e-cad4-4a9c-9867-b69846a1f4b7 admin admin] PATH_INFO: `/v3/users/5533dca4f7554684a6ef3d70ecf16b2c` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:07.400274 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e19c25e-cad4-4a9c-9867-b69846a1f4b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5533dca4f7554684a6ef3d70ecf16b2c)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:07.402224 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e19c25e-cad4-4a9c-9867-b69846a1f4b7 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:07.451584 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1e19c25e-cad4-4a9c-9867-b69846a1f4b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5533dca4f7554684a6ef3d70ecf16b2c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:07.452231 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1e19c25e-cad4-4a9c-9867-b69846a1f4b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5533dca4f7554684a6ef3d70ecf16b2c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:07.459151 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1e19c25e-cad4-4a9c-9867-b69846a1f4b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5533dca4f7554684a6ef3d70ecf16b2c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:07.472078 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1e19c25e-cad4-4a9c-9867-b69846a1f4b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5533dca4f7554684a6ef3d70ecf16b2c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:07.481106 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1e19c25e-cad4-4a9c-9867-b69846a1f4b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5533dca4f7554684a6ef3d70ecf16b2c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:07.487104 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1419/2845] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:35:07 2022] DELETE /identity/v3/users/5533dca4f7554684a6ef3d70ecf16b2c => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:35:07.768558 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58de410b-e50e-4c71-994d-5f33d9db8d1b admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:07.770231 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-58de410b-e50e-4c71-994d-5f33d9db8d1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:35:07.784742 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58de410b-e50e-4c71-994d-5f33d9db8d1b admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:07.848547 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22db965-2575-4991-8444-5699a1f300c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:07.849119 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22db965-2575-4991-8444-5699a1f300c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:07.849408 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22db965-2575-4991-8444-5699a1f300c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:07.849656 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22db965-2575-4991-8444-5699a1f300c7 admin admin] PATH_INFO: `/v3/projects/6d168e52e832405296b0692ae8df6279` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:07.850845 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22db965-2575-4991-8444-5699a1f300c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d168e52e832405296b0692ae8df6279)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:07.852781 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22db965-2575-4991-8444-5699a1f300c7 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:07.902702 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d22db965-2575-4991-8444-5699a1f300c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d168e52e832405296b0692ae8df6279', 'request_id': 'req-d22db965-2575-4991-8444-5699a1f300c7'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:07.908578 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d22db965-2575-4991-8444-5699a1f300c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d168e52e832405296b0692ae8df6279', 'request_id': 'req-d22db965-2575-4991-8444-5699a1f300c7'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:07.918971 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d22db965-2575-4991-8444-5699a1f300c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d168e52e832405296b0692ae8df6279', 'request_id': 'req-d22db965-2575-4991-8444-5699a1f300c7'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:07.931941 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d22db965-2575-4991-8444-5699a1f300c7 admin admin] The token cache is being invalidate because project 6d168e52e832405296b0692ae8df6279 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:35:07.932311 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d22db965-2575-4991-8444-5699a1f300c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:07.932956 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d22db965-2575-4991-8444-5699a1f300c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:07.934475 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1427/2846] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:35:07 2022] DELETE /identity/v3/projects/6d168e52e832405296b0692ae8df6279 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:35:07.939552 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59e5ed5-6446-4d5a-bd59-c6896d8f337c None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:07.940105 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59e5ed5-6446-4d5a-bd59-c6896d8f337c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:07.940231 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59e5ed5-6446-4d5a-bd59-c6896d8f337c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:07.964009 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-b59e5ed5-6446-4d5a-bd59-c6896d8f337c None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:35:07.965281 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b59e5ed5-6446-4d5a-bd59-c6896d8f337c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:35:08.030929 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1420/2847] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:35:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:35:08.039167 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eda8349-47e9-4d38-acf7-ffab4754ef12 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:08.039446 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eda8349-47e9-4d38-acf7-ffab4754ef12 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:08.039721 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eda8349-47e9-4d38-acf7-ffab4754ef12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:08.063359 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-7eda8349-47e9-4d38-acf7-ffab4754ef12 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:35:08.064575 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-7eda8349-47e9-4d38-acf7-ffab4754ef12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:35:08.075363 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1428/2848] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:35:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:35:08.079894 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59e5ed5-6446-4d5a-bd59-c6896d8f337c None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:08.089968 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59e5ed5-6446-4d5a-bd59-c6896d8f337c None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:08.095298 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32508a02-5bfd-43c8-b3f1-7ba38a564eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:08.095907 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32508a02-5bfd-43c8-b3f1-7ba38a564eb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:08.096180 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32508a02-5bfd-43c8-b3f1-7ba38a564eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:08.096445 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32508a02-5bfd-43c8-b3f1-7ba38a564eb5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:08.097124 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32508a02-5bfd-43c8-b3f1-7ba38a564eb5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:35:08.097419 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32508a02-5bfd-43c8-b3f1-7ba38a564eb5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:08.099072 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32508a02-5bfd-43c8-b3f1-7ba38a564eb5 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:08.114351 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1421/2849] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 6 16:35:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:35:08.118398 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eda8349-47e9-4d38-acf7-ffab4754ef12 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:08.128126 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eda8349-47e9-4d38-acf7-ffab4754ef12 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:08.133959 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de635fe3-dce0-444f-8572-05e1ce1b39da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:08.134568 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de635fe3-dce0-444f-8572-05e1ce1b39da admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:08.134848 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de635fe3-dce0-444f-8572-05e1ce1b39da admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:08.135120 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de635fe3-dce0-444f-8572-05e1ce1b39da admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:08.135771 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de635fe3-dce0-444f-8572-05e1ce1b39da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:08.137731 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de635fe3-dce0-444f-8572-05e1ce1b39da admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:08.167658 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1429/2850] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:35:08 2022] POST /identity/v3/projects => generated 369 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:35:08.171986 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32508a02-5bfd-43c8-b3f1-7ba38a564eb5 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:08.182408 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32508a02-5bfd-43c8-b3f1-7ba38a564eb5 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:08.217655 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:35:08.243539 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66df235-28a3-4c6b-bbfc-6e61bf9d2769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:08.244152 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66df235-28a3-4c6b-bbfc-6e61bf9d2769 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:08.244436 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66df235-28a3-4c6b-bbfc-6e61bf9d2769 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:08.244725 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66df235-28a3-4c6b-bbfc-6e61bf9d2769 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:08.245405 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66df235-28a3-4c6b-bbfc-6e61bf9d2769 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:08.247324 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66df235-28a3-4c6b-bbfc-6e61bf9d2769 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:08.276012 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1422/2851] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:35:08 2022] POST /identity/v3/users => generated 285 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:35:08.276279 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de635fe3-dce0-444f-8572-05e1ce1b39da admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:08.287237 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de635fe3-dce0-444f-8572-05e1ce1b39da admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:08.293185 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55982be2-b80c-4a23-928c-48dcbbefabfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:08.293786 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55982be2-b80c-4a23-928c-48dcbbefabfd admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:08.294063 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55982be2-b80c-4a23-928c-48dcbbefabfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:08.294334 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55982be2-b80c-4a23-928c-48dcbbefabfd admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:08.294961 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55982be2-b80c-4a23-928c-48dcbbefabfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:08.296611 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55982be2-b80c-4a23-928c-48dcbbefabfd admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:08.302271 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:35:08.308414 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1430/2852] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:35:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:35:08.312602 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66df235-28a3-4c6b-bbfc-6e61bf9d2769 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:08.323287 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66df235-28a3-4c6b-bbfc-6e61bf9d2769 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:08.328491 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea784e6-cb6b-4f06-b672-c2c4e1ae0952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:08.329131 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea784e6-cb6b-4f06-b672-c2c4e1ae0952 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:08.329406 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea784e6-cb6b-4f06-b672-c2c4e1ae0952 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:08.329673 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea784e6-cb6b-4f06-b672-c2c4e1ae0952 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:08.330314 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea784e6-cb6b-4f06-b672-c2c4e1ae0952 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:08.331925 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea784e6-cb6b-4f06-b672-c2c4e1ae0952 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:08.342501 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1423/2853] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:35:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:35:08.346967 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55982be2-b80c-4a23-928c-48dcbbefabfd admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:08.357821 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55982be2-b80c-4a23-928c-48dcbbefabfd admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:08.363639 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb29c3f-5659-4c0d-8d8b-6d3adaa35282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:08.364317 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb29c3f-5659-4c0d-8d8b-6d3adaa35282 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:08.364587 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb29c3f-5659-4c0d-8d8b-6d3adaa35282 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:08.364888 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb29c3f-5659-4c0d-8d8b-6d3adaa35282 admin admin] PATH_INFO: `/v3/projects/fa4671210e904014acee6b61a2252b46/users/dc35961f7d374145961179840e01b4d8/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:08.380963 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb29c3f-5659-4c0d-8d8b-6d3adaa35282 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa4671210e904014acee6b61a2252b46, user_id=dc35961f7d374145961179840e01b4d8, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:08.383656 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb29c3f-5659-4c0d-8d8b-6d3adaa35282 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:08.398065 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1431/2854] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:35:08 2022] PUT /identity/v3/projects/fa4671210e904014acee6b61a2252b46/users/dc35961f7d374145961179840e01b4d8/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:35:08.410352 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf1742a-9068-4bfe-8a60-8f3e0c860d2e None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:08.410590 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf1742a-9068-4bfe-8a60-8f3e0c860d2e None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:08.410924 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf1742a-9068-4bfe-8a60-8f3e0c860d2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:08.440925 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-0bf1742a-9068-4bfe-8a60-8f3e0c860d2e None None] MFA Rules not processed for user `dc35961f7d374145961179840e01b4d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:35:08.442134 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0bf1742a-9068-4bfe-8a60-8f3e0c860d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:35:08.509252 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1424/2855] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:35:08 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:35:08.514078 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb29c3f-5659-4c0d-8d8b-6d3adaa35282 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:08.580364 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb29c3f-5659-4c0d-8d8b-6d3adaa35282 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:08.585743 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81ab50d-b990-4971-a027-0ccab85f5cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:08.586346 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81ab50d-b990-4971-a027-0ccab85f5cf0 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:08.586624 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81ab50d-b990-4971-a027-0ccab85f5cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:08.586894 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81ab50d-b990-4971-a027-0ccab85f5cf0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:08.587556 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81ab50d-b990-4971-a027-0ccab85f5cf0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:08.589579 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81ab50d-b990-4971-a027-0ccab85f5cf0 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:08.618920 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1432/2856] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:35:08 2022] POST /identity/v3/projects => generated 369 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:35:08.622513 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf1742a-9068-4bfe-8a60-8f3e0c860d2e None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:08.633531 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf1742a-9068-4bfe-8a60-8f3e0c860d2e None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:08.694302 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d03e15-198a-417d-b34b-0440cc22d597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:08.694894 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d03e15-198a-417d-b34b-0440cc22d597 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:08.695150 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d03e15-198a-417d-b34b-0440cc22d597 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:08.695442 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d03e15-198a-417d-b34b-0440cc22d597 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:08.696079 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d03e15-198a-417d-b34b-0440cc22d597 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:08.698020 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d03e15-198a-417d-b34b-0440cc22d597 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:08.725869 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1425/2857] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:35:08 2022] POST /identity/v3/users => generated 291 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:35:08.731456 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81ab50d-b990-4971-a027-0ccab85f5cf0 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:08.742506 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81ab50d-b990-4971-a027-0ccab85f5cf0 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:08.747981 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c83d667-7bc6-4d56-a99e-d66ae821b73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:08.748579 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c83d667-7bc6-4d56-a99e-d66ae821b73a admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:08.748897 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c83d667-7bc6-4d56-a99e-d66ae821b73a admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:08.749169 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c83d667-7bc6-4d56-a99e-d66ae821b73a admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:08.749998 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c83d667-7bc6-4d56-a99e-d66ae821b73a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:08.751626 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c83d667-7bc6-4d56-a99e-d66ae821b73a admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:08.764011 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1433/2858] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:35:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:35:08.768943 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d03e15-198a-417d-b34b-0440cc22d597 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:08.783525 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d03e15-198a-417d-b34b-0440cc22d597 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:08.789318 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3543587b-2939-477e-87f8-60902fb6a330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:08.789977 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3543587b-2939-477e-87f8-60902fb6a330 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:08.790238 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3543587b-2939-477e-87f8-60902fb6a330 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:08.790506 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3543587b-2939-477e-87f8-60902fb6a330 admin admin] PATH_INFO: `/v3/projects/d69d584618cf43f280f8f5309928f045/users/414eead6579d44d5a2da9b049e2bce9f/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:08.807543 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3543587b-2939-477e-87f8-60902fb6a330 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d69d584618cf43f280f8f5309928f045, user_id=414eead6579d44d5a2da9b049e2bce9f, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:08.810272 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3543587b-2939-477e-87f8-60902fb6a330 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:08.825222 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1426/2859] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:35:08 2022] PUT /identity/v3/projects/d69d584618cf43f280f8f5309928f045/users/414eead6579d44d5a2da9b049e2bce9f/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:35:08.829683 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c83d667-7bc6-4d56-a99e-d66ae821b73a admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:08.840659 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c83d667-7bc6-4d56-a99e-d66ae821b73a admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:08.913596 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410890b9-9491-424e-a0bc-764c56b408c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:08.914131 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410890b9-9491-424e-a0bc-764c56b408c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:08.914407 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410890b9-9491-424e-a0bc-764c56b408c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:08.914678 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410890b9-9491-424e-a0bc-764c56b408c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:08.915447 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410890b9-9491-424e-a0bc-764c56b408c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:08.917396 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410890b9-9491-424e-a0bc-764c56b408c7 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:08.929063 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1434/2860] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:35:08 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:35:08.933264 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3543587b-2939-477e-87f8-60902fb6a330 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:08.946251 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3543587b-2939-477e-87f8-60902fb6a330 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:08.952219 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87813a65-6ad6-455f-8c49-93ee0e848c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:08.952912 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87813a65-6ad6-455f-8c49-93ee0e848c33 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:08.953189 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87813a65-6ad6-455f-8c49-93ee0e848c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:08.953454 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87813a65-6ad6-455f-8c49-93ee0e848c33 admin admin] PATH_INFO: `/v3/projects/d69d584618cf43f280f8f5309928f045/users/414eead6579d44d5a2da9b049e2bce9f/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:08.955620 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87813a65-6ad6-455f-8c49-93ee0e848c33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d69d584618cf43f280f8f5309928f045, user_id=414eead6579d44d5a2da9b049e2bce9f, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:08.958306 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87813a65-6ad6-455f-8c49-93ee0e848c33 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:08.964408 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:35:08.970802 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1427/2861] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:35:08 2022] PUT /identity/v3/projects/d69d584618cf43f280f8f5309928f045/users/414eead6579d44d5a2da9b049e2bce9f/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:35:08.981314 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417d69e2-31b2-4e3c-b7ff-a476b60fa1b5 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:08.981600 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417d69e2-31b2-4e3c-b7ff-a476b60fa1b5 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:08.981877 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417d69e2-31b2-4e3c-b7ff-a476b60fa1b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:09.013037 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-417d69e2-31b2-4e3c-b7ff-a476b60fa1b5 None None] MFA Rules not processed for user `414eead6579d44d5a2da9b049e2bce9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:35:09.014315 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-417d69e2-31b2-4e3c-b7ff-a476b60fa1b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:35:09.050001 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:35:09.086941 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1435/2862] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:35:08 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:35:09.098203 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87813a65-6ad6-455f-8c49-93ee0e848c33 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:09.100162 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-87813a65-6ad6-455f-8c49-93ee0e848c33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:35:09.161565 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87813a65-6ad6-455f-8c49-93ee0e848c33 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:09.165818 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cca35e-8206-429f-94b4-a4fd2ce7abc4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:09.166649 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cca35e-8206-429f-94b4-a4fd2ce7abc4 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:09.166940 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cca35e-8206-429f-94b4-a4fd2ce7abc4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:09.167212 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cca35e-8206-429f-94b4-a4fd2ce7abc4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:09.179235 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cca35e-8206-429f-94b4-a4fd2ce7abc4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:09.180672 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cca35e-8206-429f-94b4-a4fd2ce7abc4 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:09.190402 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1428/2863] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:35:09 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:35:10.617845 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417d69e2-31b2-4e3c-b7ff-a476b60fa1b5 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:10.695546 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417d69e2-31b2-4e3c-b7ff-a476b60fa1b5 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:10.701022 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f48729d-3ce2-4a3a-8993-774b9b608a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:10.701662 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f48729d-3ce2-4a3a-8993-774b9b608a3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:10.701941 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f48729d-3ce2-4a3a-8993-774b9b608a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:10.702211 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f48729d-3ce2-4a3a-8993-774b9b608a3c admin admin] PATH_INFO: `/v3/users/dc35961f7d374145961179840e01b4d8` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:10.703349 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f48729d-3ce2-4a3a-8993-774b9b608a3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc35961f7d374145961179840e01b4d8)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:10.705338 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f48729d-3ce2-4a3a-8993-774b9b608a3c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:10.761624 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-1f48729d-3ce2-4a3a-8993-774b9b608a3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc35961f7d374145961179840e01b4d8'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:10.762622 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-1f48729d-3ce2-4a3a-8993-774b9b608a3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc35961f7d374145961179840e01b4d8'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:10.773710 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-1f48729d-3ce2-4a3a-8993-774b9b608a3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc35961f7d374145961179840e01b4d8'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:10.787228 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-1f48729d-3ce2-4a3a-8993-774b9b608a3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc35961f7d374145961179840e01b4d8'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:10.788008 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-1f48729d-3ce2-4a3a-8993-774b9b608a3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc35961f7d374145961179840e01b4d8'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:10.800577 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1436/2864] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:35:10 2022] DELETE /identity/v3/users/dc35961f7d374145961179840e01b4d8 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:35:10.805398 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eda087d-0a09-41fe-ab7b-68f8d02e5d06 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:10.805935 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eda087d-0a09-41fe-ab7b-68f8d02e5d06 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:10.806066 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eda087d-0a09-41fe-ab7b-68f8d02e5d06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:10.825046 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-5eda087d-0a09-41fe-ab7b-68f8d02e5d06 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:35:10.826260 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5eda087d-0a09-41fe-ab7b-68f8d02e5d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:35:10.886763 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1429/2865] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:35:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:35:10.896874 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f48729d-3ce2-4a3a-8993-774b9b608a3c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:10.898508 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1f48729d-3ce2-4a3a-8993-774b9b608a3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:35:10.966921 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f48729d-3ce2-4a3a-8993-774b9b608a3c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:10.970985 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763fd3f0-ea66-4fde-8136-80b69d974536 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:10.971592 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763fd3f0-ea66-4fde-8136-80b69d974536 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:10.971863 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763fd3f0-ea66-4fde-8136-80b69d974536 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:10.972133 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763fd3f0-ea66-4fde-8136-80b69d974536 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:10.980113 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:35:10.985713 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763fd3f0-ea66-4fde-8136-80b69d974536 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:10.987740 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763fd3f0-ea66-4fde-8136-80b69d974536 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:10.999134 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1437/2866] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:35:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:35:11.303296 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eda087d-0a09-41fe-ab7b-68f8d02e5d06 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:11.305045 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5eda087d-0a09-41fe-ab7b-68f8d02e5d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:35:11.320203 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eda087d-0a09-41fe-ab7b-68f8d02e5d06 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:11.325587 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43888f4-5a04-4a38-910f-16e1ddcdc1fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:11.326326 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43888f4-5a04-4a38-910f-16e1ddcdc1fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:11.326645 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43888f4-5a04-4a38-910f-16e1ddcdc1fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:11.326966 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43888f4-5a04-4a38-910f-16e1ddcdc1fd admin admin] PATH_INFO: `/v3/projects/fa4671210e904014acee6b61a2252b46` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:11.328375 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43888f4-5a04-4a38-910f-16e1ddcdc1fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa4671210e904014acee6b61a2252b46)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:11.330844 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43888f4-5a04-4a38-910f-16e1ddcdc1fd admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:11.384891 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e43888f4-5a04-4a38-910f-16e1ddcdc1fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa4671210e904014acee6b61a2252b46', 'request_id': 'req-e43888f4-5a04-4a38-910f-16e1ddcdc1fd'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:11.393307 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e43888f4-5a04-4a38-910f-16e1ddcdc1fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa4671210e904014acee6b61a2252b46', 'request_id': 'req-e43888f4-5a04-4a38-910f-16e1ddcdc1fd'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:11.400552 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e43888f4-5a04-4a38-910f-16e1ddcdc1fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa4671210e904014acee6b61a2252b46', 'request_id': 'req-e43888f4-5a04-4a38-910f-16e1ddcdc1fd'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:11.414953 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e43888f4-5a04-4a38-910f-16e1ddcdc1fd admin admin] The token cache is being invalidate because project fa4671210e904014acee6b61a2252b46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:35:11.415408 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e43888f4-5a04-4a38-910f-16e1ddcdc1fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:11.416683 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e43888f4-5a04-4a38-910f-16e1ddcdc1fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:11.418600 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1430/2867] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:35:11 2022] DELETE /identity/v3/projects/fa4671210e904014acee6b61a2252b46 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:35:11.423351 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763fd3f0-ea66-4fde-8136-80b69d974536 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:11.433990 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763fd3f0-ea66-4fde-8136-80b69d974536 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:11.496493 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1276e71-29e1-4e3b-b933-4ba3d9b9bca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:11.497150 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1276e71-29e1-4e3b-b933-4ba3d9b9bca0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:11.497429 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1276e71-29e1-4e3b-b933-4ba3d9b9bca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:11.497702 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1276e71-29e1-4e3b-b933-4ba3d9b9bca0 admin admin] PATH_INFO: `/v3/users/414eead6579d44d5a2da9b049e2bce9f` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:11.498827 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1276e71-29e1-4e3b-b933-4ba3d9b9bca0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=414eead6579d44d5a2da9b049e2bce9f)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:11.500778 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1276e71-29e1-4e3b-b933-4ba3d9b9bca0 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:11.552714 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a1276e71-29e1-4e3b-b933-4ba3d9b9bca0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '414eead6579d44d5a2da9b049e2bce9f'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:11.553547 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a1276e71-29e1-4e3b-b933-4ba3d9b9bca0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '414eead6579d44d5a2da9b049e2bce9f'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:11.561099 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a1276e71-29e1-4e3b-b933-4ba3d9b9bca0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '414eead6579d44d5a2da9b049e2bce9f'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:11.573256 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a1276e71-29e1-4e3b-b933-4ba3d9b9bca0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '414eead6579d44d5a2da9b049e2bce9f'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:11.574045 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a1276e71-29e1-4e3b-b933-4ba3d9b9bca0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '414eead6579d44d5a2da9b049e2bce9f'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:11.592176 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1438/2868] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:35:11 2022] DELETE /identity/v3/users/414eead6579d44d5a2da9b049e2bce9f => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:35:11.902470 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43888f4-5a04-4a38-910f-16e1ddcdc1fd admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:11.904090 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e43888f4-5a04-4a38-910f-16e1ddcdc1fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:35:11.920748 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43888f4-5a04-4a38-910f-16e1ddcdc1fd admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:11.945485 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:35:11.993737 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc61869d-3750-42d4-9ae6-511ad289cf69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:11.994380 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc61869d-3750-42d4-9ae6-511ad289cf69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:11.994662 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc61869d-3750-42d4-9ae6-511ad289cf69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:11.994932 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc61869d-3750-42d4-9ae6-511ad289cf69 admin admin] PATH_INFO: `/v3/projects/d69d584618cf43f280f8f5309928f045` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:11.996101 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc61869d-3750-42d4-9ae6-511ad289cf69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d69d584618cf43f280f8f5309928f045)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:11.998063 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc61869d-3750-42d4-9ae6-511ad289cf69 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:12.058389 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-dc61869d-3750-42d4-9ae6-511ad289cf69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd69d584618cf43f280f8f5309928f045', 'request_id': 'req-dc61869d-3750-42d4-9ae6-511ad289cf69'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:12.067380 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-dc61869d-3750-42d4-9ae6-511ad289cf69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd69d584618cf43f280f8f5309928f045', 'request_id': 'req-dc61869d-3750-42d4-9ae6-511ad289cf69'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:12.074792 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-dc61869d-3750-42d4-9ae6-511ad289cf69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd69d584618cf43f280f8f5309928f045', 'request_id': 'req-dc61869d-3750-42d4-9ae6-511ad289cf69'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:12.089035 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-dc61869d-3750-42d4-9ae6-511ad289cf69 admin admin] The token cache is being invalidate because project d69d584618cf43f280f8f5309928f045 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:35:12.089410 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-dc61869d-3750-42d4-9ae6-511ad289cf69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:12.090129 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-dc61869d-3750-42d4-9ae6-511ad289cf69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:12.092079 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1431/2869] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:35:11 2022] DELETE /identity/v3/projects/d69d584618cf43f280f8f5309928f045 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:35:12.097349 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc8d2f8-3229-4f35-b3ad-75a57fdab444 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:12.098044 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc8d2f8-3229-4f35-b3ad-75a57fdab444 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:12.098044 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc8d2f8-3229-4f35-b3ad-75a57fdab444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:12.118897 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-9dc8d2f8-3229-4f35-b3ad-75a57fdab444 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:35:12.120105 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-9dc8d2f8-3229-4f35-b3ad-75a57fdab444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:35:12.203193 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1439/2870] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:35:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:35:12.212132 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a459d-d676-46b9-bce8-b24e03e0a6c7 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:12.212541 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a459d-d676-46b9-bce8-b24e03e0a6c7 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:12.212962 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a459d-d676-46b9-bce8-b24e03e0a6c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:12.245133 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-f41a459d-d676-46b9-bce8-b24e03e0a6c7 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:35:12.246591 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f41a459d-d676-46b9-bce8-b24e03e0a6c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:35:12.261703 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1432/2871] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:35:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:35:12.269576 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc8d2f8-3229-4f35-b3ad-75a57fdab444 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:12.286045 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc8d2f8-3229-4f35-b3ad-75a57fdab444 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:12.291008 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd6e6a9-5b0a-4611-920d-86aa88f1849e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:12.291927 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd6e6a9-5b0a-4611-920d-86aa88f1849e admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:12.292197 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd6e6a9-5b0a-4611-920d-86aa88f1849e admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:12.292462 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd6e6a9-5b0a-4611-920d-86aa88f1849e admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:12.293146 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd6e6a9-5b0a-4611-920d-86aa88f1849e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:35:12.293439 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd6e6a9-5b0a-4611-920d-86aa88f1849e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:12.295049 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd6e6a9-5b0a-4611-920d-86aa88f1849e admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:12.304399 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:35:12.308056 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1440/2872] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 6 16:35:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:35:12.314354 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41a459d-d676-46b9-bce8-b24e03e0a6c7 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:12.324219 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41a459d-d676-46b9-bce8-b24e03e0a6c7 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:12.329713 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d910b565-56a6-4c5e-932a-f413f5a7b3a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:12.330345 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d910b565-56a6-4c5e-932a-f413f5a7b3a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:12.330614 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d910b565-56a6-4c5e-932a-f413f5a7b3a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:12.330879 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d910b565-56a6-4c5e-932a-f413f5a7b3a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:12.331502 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d910b565-56a6-4c5e-932a-f413f5a7b3a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:12.333460 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d910b565-56a6-4c5e-932a-f413f5a7b3a8 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:12.362369 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1433/2873] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:35:12 2022] POST /identity/v3/projects => generated 393 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:35:12.366755 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd6e6a9-5b0a-4611-920d-86aa88f1849e admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:12.377321 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd6e6a9-5b0a-4611-920d-86aa88f1849e admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:12.438584 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589ab417-8c11-4d84-b2f5-73e412aa8882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:12.439356 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ab417-8c11-4d84-b2f5-73e412aa8882 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:12.439634 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ab417-8c11-4d84-b2f5-73e412aa8882 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:12.439902 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ab417-8c11-4d84-b2f5-73e412aa8882 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:12.440568 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589ab417-8c11-4d84-b2f5-73e412aa8882 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:12.442744 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589ab417-8c11-4d84-b2f5-73e412aa8882 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:12.470014 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1441/2874] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:35:12 2022] POST /identity/v3/users => generated 297 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:35:12.471340 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d910b565-56a6-4c5e-932a-f413f5a7b3a8 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:12.482094 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d910b565-56a6-4c5e-932a-f413f5a7b3a8 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:12.486959 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729538f0-59ea-4fb5-813a-450949dd9025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:12.487597 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729538f0-59ea-4fb5-813a-450949dd9025 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:12.487869 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729538f0-59ea-4fb5-813a-450949dd9025 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:12.488178 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729538f0-59ea-4fb5-813a-450949dd9025 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:12.488844 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729538f0-59ea-4fb5-813a-450949dd9025 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:12.490498 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729538f0-59ea-4fb5-813a-450949dd9025 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:12.500546 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1434/2875] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:35:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:35:12.505006 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589ab417-8c11-4d84-b2f5-73e412aa8882 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:12.515889 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589ab417-8c11-4d84-b2f5-73e412aa8882 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:12.522517 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b789841-19a2-40a8-be37-d455d0570389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:12.523093 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b789841-19a2-40a8-be37-d455d0570389 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:12.523453 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b789841-19a2-40a8-be37-d455d0570389 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:12.523727 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b789841-19a2-40a8-be37-d455d0570389 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:12.524270 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b789841-19a2-40a8-be37-d455d0570389 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:12.525917 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b789841-19a2-40a8-be37-d455d0570389 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:12.537207 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1442/2876] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:35:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:35:12.541994 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729538f0-59ea-4fb5-813a-450949dd9025 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:12.552105 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729538f0-59ea-4fb5-813a-450949dd9025 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:12.557545 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772b850d-f433-4f78-b05a-7f112f48c1a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:12.558229 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772b850d-f433-4f78-b05a-7f112f48c1a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:12.558499 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772b850d-f433-4f78-b05a-7f112f48c1a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:12.558763 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772b850d-f433-4f78-b05a-7f112f48c1a3 admin admin] PATH_INFO: `/v3/projects/530610350c4e45c386a6e3aeec55adb2/users/c3dc6bb261b24af390ad8ae0a6f60619/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:12.574358 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772b850d-f433-4f78-b05a-7f112f48c1a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=530610350c4e45c386a6e3aeec55adb2, user_id=c3dc6bb261b24af390ad8ae0a6f60619, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:12.576973 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772b850d-f433-4f78-b05a-7f112f48c1a3 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:12.591027 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1435/2877] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:35:12 2022] PUT /identity/v3/projects/530610350c4e45c386a6e3aeec55adb2/users/c3dc6bb261b24af390ad8ae0a6f60619/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:35:12.658961 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6a10dd-0f0f-41e4-8d2d-33784632b8d1 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:12.659262 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6a10dd-0f0f-41e4-8d2d-33784632b8d1 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:12.659493 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6a10dd-0f0f-41e4-8d2d-33784632b8d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:12.689314 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-ba6a10dd-0f0f-41e4-8d2d-33784632b8d1 None None] MFA Rules not processed for user `c3dc6bb261b24af390ad8ae0a6f60619`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:35:12.690525 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ba6a10dd-0f0f-41e4-8d2d-33784632b8d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:35:12.758753 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1443/2878] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:35:12 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:35:12.763781 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772b850d-f433-4f78-b05a-7f112f48c1a3 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:12.819720 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:35:12.834958 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772b850d-f433-4f78-b05a-7f112f48c1a3 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:12.840487 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8221ce25-7e84-45f9-b9d0-0cce39361a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:12.841142 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8221ce25-7e84-45f9-b9d0-0cce39361a85 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:12.841422 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8221ce25-7e84-45f9-b9d0-0cce39361a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:12.841699 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8221ce25-7e84-45f9-b9d0-0cce39361a85 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:12.842346 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8221ce25-7e84-45f9-b9d0-0cce39361a85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:12.844279 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8221ce25-7e84-45f9-b9d0-0cce39361a85 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:12.875908 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1436/2879] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:35:12 2022] POST /identity/v3/projects => generated 393 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:35:12.880223 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6a10dd-0f0f-41e4-8d2d-33784632b8d1 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:12.891755 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6a10dd-0f0f-41e4-8d2d-33784632b8d1 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:12.958177 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d0a557-37fe-44f7-8cb2-65f4bc557015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:12.958791 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d0a557-37fe-44f7-8cb2-65f4bc557015 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:12.959069 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d0a557-37fe-44f7-8cb2-65f4bc557015 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:12.959430 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d0a557-37fe-44f7-8cb2-65f4bc557015 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:12.960073 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d0a557-37fe-44f7-8cb2-65f4bc557015 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:12.962062 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d0a557-37fe-44f7-8cb2-65f4bc557015 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:12.979495 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:35:12.986278 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1444/2880] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:35:12 2022] POST /identity/v3/users => generated 303 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:35:12.990488 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8221ce25-7e84-45f9-b9d0-0cce39361a85 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:13.001487 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8221ce25-7e84-45f9-b9d0-0cce39361a85 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:13.011067 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e94e4f-7199-4ac4-9c5b-640f0d25672e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:13.011681 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e94e4f-7199-4ac4-9c5b-640f0d25672e admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:13.011955 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e94e4f-7199-4ac4-9c5b-640f0d25672e admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:13.012218 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e94e4f-7199-4ac4-9c5b-640f0d25672e admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:13.012900 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e94e4f-7199-4ac4-9c5b-640f0d25672e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:13.014720 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e94e4f-7199-4ac4-9c5b-640f0d25672e admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:13.026589 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1437/2881] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:35:12 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:35:13.031091 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d0a557-37fe-44f7-8cb2-65f4bc557015 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:13.042919 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d0a557-37fe-44f7-8cb2-65f4bc557015 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:13.048581 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b366a26-ba76-4a89-b89e-b923716c2176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:13.049264 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b366a26-ba76-4a89-b89e-b923716c2176 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:13.049540 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b366a26-ba76-4a89-b89e-b923716c2176 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:13.049814 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b366a26-ba76-4a89-b89e-b923716c2176 admin admin] PATH_INFO: `/v3/projects/123458ac7cd84655bec891ebae5716d3/users/5e28fd1bab554012be8499e19e2a8955/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:13.067683 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b366a26-ba76-4a89-b89e-b923716c2176 admin admin] RBAC: Authorizing `identity:create_grant(project_id=123458ac7cd84655bec891ebae5716d3, user_id=5e28fd1bab554012be8499e19e2a8955, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:13.070477 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b366a26-ba76-4a89-b89e-b923716c2176 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:13.086298 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1445/2882] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:35:13 2022] PUT /identity/v3/projects/123458ac7cd84655bec891ebae5716d3/users/5e28fd1bab554012be8499e19e2a8955/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:35:13.090442 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e94e4f-7199-4ac4-9c5b-640f0d25672e admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:13.100966 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e94e4f-7199-4ac4-9c5b-640f0d25672e admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:13.163468 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28e4814-766a-47f5-8e83-42bc662a8654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:13.164093 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28e4814-766a-47f5-8e83-42bc662a8654 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:13.164366 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28e4814-766a-47f5-8e83-42bc662a8654 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:13.164633 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28e4814-766a-47f5-8e83-42bc662a8654 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:13.165309 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28e4814-766a-47f5-8e83-42bc662a8654 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:13.167376 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28e4814-766a-47f5-8e83-42bc662a8654 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:13.181209 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1438/2883] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:35:13 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 16:35:13.185313 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b366a26-ba76-4a89-b89e-b923716c2176 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:13.196152 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b366a26-ba76-4a89-b89e-b923716c2176 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:13.201313 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6425cdbe-126c-473f-921d-92c243aef465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:13.201944 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6425cdbe-126c-473f-921d-92c243aef465 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:13.202237 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6425cdbe-126c-473f-921d-92c243aef465 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:13.202488 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6425cdbe-126c-473f-921d-92c243aef465 admin admin] PATH_INFO: `/v3/projects/123458ac7cd84655bec891ebae5716d3/users/5e28fd1bab554012be8499e19e2a8955/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:13.204501 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6425cdbe-126c-473f-921d-92c243aef465 admin admin] RBAC: Authorizing `identity:create_grant(project_id=123458ac7cd84655bec891ebae5716d3, user_id=5e28fd1bab554012be8499e19e2a8955, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:13.207163 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6425cdbe-126c-473f-921d-92c243aef465 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:13.218373 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1446/2884] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:35:13 2022] PUT /identity/v3/projects/123458ac7cd84655bec891ebae5716d3/users/5e28fd1bab554012be8499e19e2a8955/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:35:13.228851 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba83574-4272-42d2-835c-a8ce4ad89d02 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:13.229139 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba83574-4272-42d2-835c-a8ce4ad89d02 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:13.229419 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba83574-4272-42d2-835c-a8ce4ad89d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:13.260927 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-9ba83574-4272-42d2-835c-a8ce4ad89d02 None None] MFA Rules not processed for user `5e28fd1bab554012be8499e19e2a8955`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:35:13.262703 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-9ba83574-4272-42d2-835c-a8ce4ad89d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:35:13.338205 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1439/2885] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:35:13 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:35:13.349300 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6425cdbe-126c-473f-921d-92c243aef465 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:13.351275 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6425cdbe-126c-473f-921d-92c243aef465 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:35:13.434960 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6425cdbe-126c-473f-921d-92c243aef465 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:13.441798 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e3f574-84ce-45ca-9f3e-54b365ba90d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:13.442439 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e3f574-84ce-45ca-9f3e-54b365ba90d7 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:13.442716 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e3f574-84ce-45ca-9f3e-54b365ba90d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:13.442982 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e3f574-84ce-45ca-9f3e-54b365ba90d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:13.454238 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e3f574-84ce-45ca-9f3e-54b365ba90d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:13.455717 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e3f574-84ce-45ca-9f3e-54b365ba90d7 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:13.467537 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1447/2886] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:35:13 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:35:13.529683 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba83574-4272-42d2-835c-a8ce4ad89d02 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:13.531293 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-9ba83574-4272-42d2-835c-a8ce4ad89d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:35:13.545274 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba83574-4272-42d2-835c-a8ce4ad89d02 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:13.549753 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de90b66-eda5-4f53-9b40-39d2c677b219 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:13.550357 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de90b66-eda5-4f53-9b40-39d2c677b219 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:13.550640 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de90b66-eda5-4f53-9b40-39d2c677b219 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:13.550909 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de90b66-eda5-4f53-9b40-39d2c677b219 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:13.602795 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:35:13.619898 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de90b66-eda5-4f53-9b40-39d2c677b219 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:13.621479 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de90b66-eda5-4f53-9b40-39d2c677b219 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:13.631155 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1440/2887] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:35:13 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:35:17.033908 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e3f574-84ce-45ca-9f3e-54b365ba90d7 service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:17.103340 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e3f574-84ce-45ca-9f3e-54b365ba90d7 service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:17.107918 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db8777f-3f18-4e16-b7d0-d301b76219af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:17.108554 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db8777f-3f18-4e16-b7d0-d301b76219af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:17.108863 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db8777f-3f18-4e16-b7d0-d301b76219af admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:17.109139 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db8777f-3f18-4e16-b7d0-d301b76219af admin admin] PATH_INFO: `/v3/users/c3dc6bb261b24af390ad8ae0a6f60619` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:17.110300 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db8777f-3f18-4e16-b7d0-d301b76219af admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3dc6bb261b24af390ad8ae0a6f60619)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:17.112244 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db8777f-3f18-4e16-b7d0-d301b76219af admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:17.174962 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:35:17.177053 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8db8777f-3f18-4e16-b7d0-d301b76219af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3dc6bb261b24af390ad8ae0a6f60619'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:17.177987 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8db8777f-3f18-4e16-b7d0-d301b76219af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3dc6bb261b24af390ad8ae0a6f60619'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:17.187783 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8db8777f-3f18-4e16-b7d0-d301b76219af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3dc6bb261b24af390ad8ae0a6f60619'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:17.201622 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8db8777f-3f18-4e16-b7d0-d301b76219af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3dc6bb261b24af390ad8ae0a6f60619'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:17.202374 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8db8777f-3f18-4e16-b7d0-d301b76219af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3dc6bb261b24af390ad8ae0a6f60619'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:17.216695 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1448/2888] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:35:17 2022] DELETE /identity/v3/users/c3dc6bb261b24af390ad8ae0a6f60619 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:35:17.221617 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c9b42-1aa3-45e0-9cc5-5b1c440352c3 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:17.222158 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c9b42-1aa3-45e0-9cc5-5b1c440352c3 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:17.222284 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c9b42-1aa3-45e0-9cc5-5b1c440352c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:17.243562 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-a65c9b42-1aa3-45e0-9cc5-5b1c440352c3 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:35:17.244798 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a65c9b42-1aa3-45e0-9cc5-5b1c440352c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:35:17.314828 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1441/2889] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:35:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:35:17.324711 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db8777f-3f18-4e16-b7d0-d301b76219af admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:17.326184 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-8db8777f-3f18-4e16-b7d0-d301b76219af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:35:17.385959 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db8777f-3f18-4e16-b7d0-d301b76219af admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:17.389795 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ed4c0c-51b8-40bd-8ba4-9260f81281f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:17.390419 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed4c0c-51b8-40bd-8ba4-9260f81281f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:17.390692 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed4c0c-51b8-40bd-8ba4-9260f81281f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:17.390966 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed4c0c-51b8-40bd-8ba4-9260f81281f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:17.401740 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ed4c0c-51b8-40bd-8ba4-9260f81281f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:17.403198 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ed4c0c-51b8-40bd-8ba4-9260f81281f6 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:17.413405 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1449/2890] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:35:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:35:17.705630 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65c9b42-1aa3-45e0-9cc5-5b1c440352c3 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:17.707377 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a65c9b42-1aa3-45e0-9cc5-5b1c440352c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:35:17.722984 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65c9b42-1aa3-45e0-9cc5-5b1c440352c3 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:17.730126 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dfce2a-8660-42ea-9d27-5eb170b5165f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:17.730835 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dfce2a-8660-42ea-9d27-5eb170b5165f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:17.731105 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dfce2a-8660-42ea-9d27-5eb170b5165f admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:17.731307 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dfce2a-8660-42ea-9d27-5eb170b5165f admin admin] PATH_INFO: `/v3/projects/530610350c4e45c386a6e3aeec55adb2` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:17.732638 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dfce2a-8660-42ea-9d27-5eb170b5165f admin admin] RBAC: Authorizing `identity:delete_project(project_id=530610350c4e45c386a6e3aeec55adb2)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:17.734527 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dfce2a-8660-42ea-9d27-5eb170b5165f admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:17.785785 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-17dfce2a-8660-42ea-9d27-5eb170b5165f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '530610350c4e45c386a6e3aeec55adb2', 'request_id': 'req-17dfce2a-8660-42ea-9d27-5eb170b5165f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:17.799609 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-17dfce2a-8660-42ea-9d27-5eb170b5165f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '530610350c4e45c386a6e3aeec55adb2', 'request_id': 'req-17dfce2a-8660-42ea-9d27-5eb170b5165f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:17.806280 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-17dfce2a-8660-42ea-9d27-5eb170b5165f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '530610350c4e45c386a6e3aeec55adb2', 'request_id': 'req-17dfce2a-8660-42ea-9d27-5eb170b5165f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:17.818499 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-17dfce2a-8660-42ea-9d27-5eb170b5165f admin admin] The token cache is being invalidate because project 530610350c4e45c386a6e3aeec55adb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:35:17.818813 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-17dfce2a-8660-42ea-9d27-5eb170b5165f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:17.819666 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-17dfce2a-8660-42ea-9d27-5eb170b5165f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:17.821305 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1442/2891] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:35:17 2022] DELETE /identity/v3/projects/530610350c4e45c386a6e3aeec55adb2 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:35:17.825552 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ed4c0c-51b8-40bd-8ba4-9260f81281f6 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:17.836867 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ed4c0c-51b8-40bd-8ba4-9260f81281f6 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:17.898277 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b700bf7a-118b-4b0f-93f5-981a07936ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:17.899044 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b700bf7a-118b-4b0f-93f5-981a07936ee2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:17.899394 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b700bf7a-118b-4b0f-93f5-981a07936ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:17.899738 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b700bf7a-118b-4b0f-93f5-981a07936ee2 admin admin] PATH_INFO: `/v3/users/5e28fd1bab554012be8499e19e2a8955` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:17.901697 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b700bf7a-118b-4b0f-93f5-981a07936ee2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e28fd1bab554012be8499e19e2a8955)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:17.905031 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b700bf7a-118b-4b0f-93f5-981a07936ee2 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:17.960810 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b700bf7a-118b-4b0f-93f5-981a07936ee2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e28fd1bab554012be8499e19e2a8955'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:17.961487 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b700bf7a-118b-4b0f-93f5-981a07936ee2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e28fd1bab554012be8499e19e2a8955'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:17.968922 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b700bf7a-118b-4b0f-93f5-981a07936ee2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e28fd1bab554012be8499e19e2a8955'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:17.976910 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:35:17.982653 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b700bf7a-118b-4b0f-93f5-981a07936ee2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e28fd1bab554012be8499e19e2a8955'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:17.983520 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b700bf7a-118b-4b0f-93f5-981a07936ee2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e28fd1bab554012be8499e19e2a8955'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:17.996622 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1450/2892] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:35:17 2022] DELETE /identity/v3/users/5e28fd1bab554012be8499e19e2a8955 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:35:18.287227 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dfce2a-8660-42ea-9d27-5eb170b5165f admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:18.288694 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-17dfce2a-8660-42ea-9d27-5eb170b5165f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:35:18.303483 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dfce2a-8660-42ea-9d27-5eb170b5165f admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:18.372381 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6ca29b-f105-4046-b61a-afeecbba3726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:18.373060 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6ca29b-f105-4046-b61a-afeecbba3726 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:18.373339 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6ca29b-f105-4046-b61a-afeecbba3726 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:18.373612 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6ca29b-f105-4046-b61a-afeecbba3726 admin admin] PATH_INFO: `/v3/projects/123458ac7cd84655bec891ebae5716d3` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:18.374781 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6ca29b-f105-4046-b61a-afeecbba3726 admin admin] RBAC: Authorizing `identity:delete_project(project_id=123458ac7cd84655bec891ebae5716d3)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:18.376709 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6ca29b-f105-4046-b61a-afeecbba3726 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:18.387992 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:35:18.428229 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-bb6ca29b-f105-4046-b61a-afeecbba3726 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '123458ac7cd84655bec891ebae5716d3', 'request_id': 'req-bb6ca29b-f105-4046-b61a-afeecbba3726'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:18.436569 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-bb6ca29b-f105-4046-b61a-afeecbba3726 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '123458ac7cd84655bec891ebae5716d3', 'request_id': 'req-bb6ca29b-f105-4046-b61a-afeecbba3726'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:18.442355 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-bb6ca29b-f105-4046-b61a-afeecbba3726 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '123458ac7cd84655bec891ebae5716d3', 'request_id': 'req-bb6ca29b-f105-4046-b61a-afeecbba3726'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:18.453798 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-bb6ca29b-f105-4046-b61a-afeecbba3726 admin admin] The token cache is being invalidate because project 123458ac7cd84655bec891ebae5716d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:35:18.454084 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-bb6ca29b-f105-4046-b61a-afeecbba3726 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:18.454762 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-bb6ca29b-f105-4046-b61a-afeecbba3726 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:18.456256 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1443/2893] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:35:18 2022] DELETE /identity/v3/projects/123458ac7cd84655bec891ebae5716d3 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:35:18.461494 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8712a63f-b373-438c-9abe-6d0010aed133 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:18.461989 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8712a63f-b373-438c-9abe-6d0010aed133 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:18.462092 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8712a63f-b373-438c-9abe-6d0010aed133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:18.482918 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-8712a63f-b373-438c-9abe-6d0010aed133 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:35:18.484260 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-8712a63f-b373-438c-9abe-6d0010aed133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:35:18.548435 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1451/2894] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:35:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:35:18.556499 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b523bb7-ac5f-4ef4-bda6-35d0a034bbab None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:18.556804 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b523bb7-ac5f-4ef4-bda6-35d0a034bbab None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:18.557080 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b523bb7-ac5f-4ef4-bda6-35d0a034bbab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:18.578442 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-7b523bb7-ac5f-4ef4-bda6-35d0a034bbab None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:35:18.579663 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7b523bb7-ac5f-4ef4-bda6-35d0a034bbab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:35:18.588563 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1444/2895] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:35:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:35:18.593267 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8712a63f-b373-438c-9abe-6d0010aed133 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:18.603776 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8712a63f-b373-438c-9abe-6d0010aed133 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:18.608778 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b072d67-9207-42c7-aeec-d6dfc5bde931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:18.609377 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b072d67-9207-42c7-aeec-d6dfc5bde931 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:18.609650 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b072d67-9207-42c7-aeec-d6dfc5bde931 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:18.609900 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b072d67-9207-42c7-aeec-d6dfc5bde931 admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:18.610575 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b072d67-9207-42c7-aeec-d6dfc5bde931 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:35:18.610868 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b072d67-9207-42c7-aeec-d6dfc5bde931 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:18.612643 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b072d67-9207-42c7-aeec-d6dfc5bde931 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:18.625040 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1452/2896] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 6 16:35:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:35:18.629776 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b523bb7-ac5f-4ef4-bda6-35d0a034bbab None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:18.639405 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b523bb7-ac5f-4ef4-bda6-35d0a034bbab None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:18.649732 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed4ebde-0f17-471e-b97d-362cb475f36e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:18.650337 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed4ebde-0f17-471e-b97d-362cb475f36e admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:18.650613 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed4ebde-0f17-471e-b97d-362cb475f36e admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:18.650877 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed4ebde-0f17-471e-b97d-362cb475f36e admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:18.651539 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed4ebde-0f17-471e-b97d-362cb475f36e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:18.653505 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed4ebde-0f17-471e-b97d-362cb475f36e admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:18.684370 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1445/2897] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:35:18 2022] POST /identity/v3/projects => generated 379 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:35:18.688780 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b072d67-9207-42c7-aeec-d6dfc5bde931 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:18.699879 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b072d67-9207-42c7-aeec-d6dfc5bde931 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:18.762225 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665ddcec-7b19-45bd-b98f-5cd0489f5026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:18.762826 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665ddcec-7b19-45bd-b98f-5cd0489f5026 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:18.763097 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665ddcec-7b19-45bd-b98f-5cd0489f5026 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:18.763360 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665ddcec-7b19-45bd-b98f-5cd0489f5026 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:18.763982 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665ddcec-7b19-45bd-b98f-5cd0489f5026 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:18.765964 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665ddcec-7b19-45bd-b98f-5cd0489f5026 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:18.787985 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1453/2898] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:35:18 2022] POST /identity/v3/users => generated 290 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:35:18.792482 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed4ebde-0f17-471e-b97d-362cb475f36e admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:18.803490 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed4ebde-0f17-471e-b97d-362cb475f36e admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:18.809213 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63a9ad0-799f-4e23-ad86-bde447d7bfea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:18.809843 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63a9ad0-799f-4e23-ad86-bde447d7bfea admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:18.810117 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63a9ad0-799f-4e23-ad86-bde447d7bfea admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:18.810383 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63a9ad0-799f-4e23-ad86-bde447d7bfea admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:18.811025 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63a9ad0-799f-4e23-ad86-bde447d7bfea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:18.812630 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63a9ad0-799f-4e23-ad86-bde447d7bfea admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:18.823311 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1446/2899] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:35:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 16:35:18.827918 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665ddcec-7b19-45bd-b98f-5cd0489f5026 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:18.838977 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665ddcec-7b19-45bd-b98f-5cd0489f5026 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:18.844453 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d4c82e-90c2-44b7-9750-6f4cc5ed27bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:18.845103 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d4c82e-90c2-44b7-9750-6f4cc5ed27bd admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:18.845390 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d4c82e-90c2-44b7-9750-6f4cc5ed27bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:18.845659 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d4c82e-90c2-44b7-9750-6f4cc5ed27bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:18.846303 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d4c82e-90c2-44b7-9750-6f4cc5ed27bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:18.848069 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d4c82e-90c2-44b7-9750-6f4cc5ed27bd admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:18.863830 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1454/2900] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:35:18 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:35:18.873074 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63a9ad0-799f-4e23-ad86-bde447d7bfea admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:18.885174 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63a9ad0-799f-4e23-ad86-bde447d7bfea admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:18.891128 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db056588-ae98-42be-b7ff-8d9a16427deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:18.891775 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db056588-ae98-42be-b7ff-8d9a16427deb admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:18.892047 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db056588-ae98-42be-b7ff-8d9a16427deb admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:18.892313 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db056588-ae98-42be-b7ff-8d9a16427deb admin admin] PATH_INFO: `/v3/projects/dea3dca0e3464a578a52cd619d5a338c/users/69e75648a9cd4ea78121ed547f7f8e07/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:18.908261 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db056588-ae98-42be-b7ff-8d9a16427deb admin admin] RBAC: Authorizing `identity:create_grant(project_id=dea3dca0e3464a578a52cd619d5a338c, user_id=69e75648a9cd4ea78121ed547f7f8e07, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:18.910907 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db056588-ae98-42be-b7ff-8d9a16427deb admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:18.925838 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1447/2901] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:35:18 2022] PUT /identity/v3/projects/dea3dca0e3464a578a52cd619d5a338c/users/69e75648a9cd4ea78121ed547f7f8e07/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:35:18.937666 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084b856d-cf95-4b59-94b1-0a7dc187789a None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:18.937931 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084b856d-cf95-4b59-94b1-0a7dc187789a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:18.938207 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084b856d-cf95-4b59-94b1-0a7dc187789a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:18.968535 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-084b856d-cf95-4b59-94b1-0a7dc187789a None None] MFA Rules not processed for user `69e75648a9cd4ea78121ed547f7f8e07`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:35:18.969799 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-084b856d-cf95-4b59-94b1-0a7dc187789a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:35:19.013889 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:35:19.043488 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1455/2902] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:35:18 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:35:19.048255 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db056588-ae98-42be-b7ff-8d9a16427deb admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:19.116254 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db056588-ae98-42be-b7ff-8d9a16427deb admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:19.121702 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78032a7e-aecb-4d58-90cf-7d47a441c818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:19.122307 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78032a7e-aecb-4d58-90cf-7d47a441c818 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:19.122586 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78032a7e-aecb-4d58-90cf-7d47a441c818 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:19.122860 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78032a7e-aecb-4d58-90cf-7d47a441c818 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:19.123506 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78032a7e-aecb-4d58-90cf-7d47a441c818 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:19.125455 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78032a7e-aecb-4d58-90cf-7d47a441c818 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:19.156057 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1448/2903] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:35:19 2022] POST /identity/v3/projects => generated 379 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:35:19.160354 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084b856d-cf95-4b59-94b1-0a7dc187789a None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:19.171773 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084b856d-cf95-4b59-94b1-0a7dc187789a None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:19.230176 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fe24e9-7760-4a20-bf11-3c950769aad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:19.230698 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fe24e9-7760-4a20-bf11-3c950769aad6 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:19.230967 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fe24e9-7760-4a20-bf11-3c950769aad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:19.231234 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fe24e9-7760-4a20-bf11-3c950769aad6 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:19.231859 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fe24e9-7760-4a20-bf11-3c950769aad6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:19.233819 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fe24e9-7760-4a20-bf11-3c950769aad6 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:19.258510 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1456/2904] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:35:19 2022] POST /identity/v3/users => generated 296 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:35:19.262740 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78032a7e-aecb-4d58-90cf-7d47a441c818 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:19.273935 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78032a7e-aecb-4d58-90cf-7d47a441c818 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:19.279210 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2254048-f74e-40d0-8556-7d1a08fe7c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:19.279978 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2254048-f74e-40d0-8556-7d1a08fe7c95 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:19.280258 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2254048-f74e-40d0-8556-7d1a08fe7c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:19.280529 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2254048-f74e-40d0-8556-7d1a08fe7c95 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:19.285281 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2254048-f74e-40d0-8556-7d1a08fe7c95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:19.285281 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2254048-f74e-40d0-8556-7d1a08fe7c95 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:19.302890 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1449/2905] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:35:19 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:35:19.307970 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fe24e9-7760-4a20-bf11-3c950769aad6 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:19.322255 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fe24e9-7760-4a20-bf11-3c950769aad6 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:19.330839 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51c4336-1506-48ea-a9eb-3da830b2b351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:19.331494 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c4336-1506-48ea-a9eb-3da830b2b351 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:19.331790 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c4336-1506-48ea-a9eb-3da830b2b351 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:19.332061 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c4336-1506-48ea-a9eb-3da830b2b351 admin admin] PATH_INFO: `/v3/projects/29506b4457d24be0b89c366ecefe2577/users/2ebd3aedbe4e413ab9e0a0754f5ea245/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:19.349991 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51c4336-1506-48ea-a9eb-3da830b2b351 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29506b4457d24be0b89c366ecefe2577, user_id=2ebd3aedbe4e413ab9e0a0754f5ea245, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:19.352881 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51c4336-1506-48ea-a9eb-3da830b2b351 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:19.367089 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1457/2906] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:35:19 2022] PUT /identity/v3/projects/29506b4457d24be0b89c366ecefe2577/users/2ebd3aedbe4e413ab9e0a0754f5ea245/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:35:19.371334 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2254048-f74e-40d0-8556-7d1a08fe7c95 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:19.389203 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2254048-f74e-40d0-8556-7d1a08fe7c95 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:19.396114 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:35:19.462171 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0f0054-413d-4950-944c-d940565c635f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:19.462782 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f0054-413d-4950-944c-d940565c635f admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:19.463058 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f0054-413d-4950-944c-d940565c635f admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:19.463345 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f0054-413d-4950-944c-d940565c635f admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:19.463986 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0f0054-413d-4950-944c-d940565c635f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:19.465639 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0f0054-413d-4950-944c-d940565c635f admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:19.477208 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1450/2907] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:35:19 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:35:19.482007 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51c4336-1506-48ea-a9eb-3da830b2b351 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:19.492736 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51c4336-1506-48ea-a9eb-3da830b2b351 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:19.497888 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5ecf2a-5761-4c51-b94a-dbe55fc18f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:19.498548 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5ecf2a-5761-4c51-b94a-dbe55fc18f54 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:19.498827 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5ecf2a-5761-4c51-b94a-dbe55fc18f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:19.499100 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5ecf2a-5761-4c51-b94a-dbe55fc18f54 admin admin] PATH_INFO: `/v3/projects/29506b4457d24be0b89c366ecefe2577/users/2ebd3aedbe4e413ab9e0a0754f5ea245/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:19.504817 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5ecf2a-5761-4c51-b94a-dbe55fc18f54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29506b4457d24be0b89c366ecefe2577, user_id=2ebd3aedbe4e413ab9e0a0754f5ea245, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:19.507689 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5ecf2a-5761-4c51-b94a-dbe55fc18f54 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:19.519013 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1458/2908] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:35:19 2022] PUT /identity/v3/projects/29506b4457d24be0b89c366ecefe2577/users/2ebd3aedbe4e413ab9e0a0754f5ea245/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:35:19.530251 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d6aac6-5f7d-4e79-a2da-8b58c0f4c303 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:19.530456 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d6aac6-5f7d-4e79-a2da-8b58c0f4c303 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:19.530943 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d6aac6-5f7d-4e79-a2da-8b58c0f4c303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:19.561954 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-c9d6aac6-5f7d-4e79-a2da-8b58c0f4c303 None None] MFA Rules not processed for user `2ebd3aedbe4e413ab9e0a0754f5ea245`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:35:19.563165 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c9d6aac6-5f7d-4e79-a2da-8b58c0f4c303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:35:19.636525 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1451/2909] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:35:19 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:35:19.648340 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5ecf2a-5761-4c51-b94a-dbe55fc18f54 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:19.650190 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6a5ecf2a-5761-4c51-b94a-dbe55fc18f54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:35:19.713906 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5ecf2a-5761-4c51-b94a-dbe55fc18f54 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:19.718092 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ae0b62-08ec-4f0c-99f4-c88bbe1c329f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:19.718703 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae0b62-08ec-4f0c-99f4-c88bbe1c329f service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:19.718979 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae0b62-08ec-4f0c-99f4-c88bbe1c329f service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:19.719247 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae0b62-08ec-4f0c-99f4-c88bbe1c329f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:19.729643 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ae0b62-08ec-4f0c-99f4-c88bbe1c329f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:19.731281 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ae0b62-08ec-4f0c-99f4-c88bbe1c329f service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:19.742156 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1459/2910] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:35:19 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:35:20.141537 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d6aac6-5f7d-4e79-a2da-8b58c0f4c303 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:20.219481 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d6aac6-5f7d-4e79-a2da-8b58c0f4c303 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:20.224646 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a266addc-5d2e-41a4-bb52-1d119fec7db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:20.225358 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a266addc-5d2e-41a4-bb52-1d119fec7db5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:20.225633 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a266addc-5d2e-41a4-bb52-1d119fec7db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:20.225901 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a266addc-5d2e-41a4-bb52-1d119fec7db5 admin admin] PATH_INFO: `/v3/users/69e75648a9cd4ea78121ed547f7f8e07` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:20.227101 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a266addc-5d2e-41a4-bb52-1d119fec7db5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69e75648a9cd4ea78121ed547f7f8e07)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:20.229034 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a266addc-5d2e-41a4-bb52-1d119fec7db5 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:20.268516 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:35:20.283091 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a266addc-5d2e-41a4-bb52-1d119fec7db5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69e75648a9cd4ea78121ed547f7f8e07'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:20.283864 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a266addc-5d2e-41a4-bb52-1d119fec7db5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69e75648a9cd4ea78121ed547f7f8e07'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:20.291806 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a266addc-5d2e-41a4-bb52-1d119fec7db5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69e75648a9cd4ea78121ed547f7f8e07'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:20.304773 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a266addc-5d2e-41a4-bb52-1d119fec7db5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69e75648a9cd4ea78121ed547f7f8e07'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:20.314860 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a266addc-5d2e-41a4-bb52-1d119fec7db5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69e75648a9cd4ea78121ed547f7f8e07'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:20.321072 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1452/2911] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:35:20 2022] DELETE /identity/v3/users/69e75648a9cd4ea78121ed547f7f8e07 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:35:20.325632 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c315300-2950-4659-b12c-25b80f0aa68f None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:20.325917 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c315300-2950-4659-b12c-25b80f0aa68f None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:20.326196 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c315300-2950-4659-b12c-25b80f0aa68f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:20.344971 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-4c315300-2950-4659-b12c-25b80f0aa68f None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:35:20.346157 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-4c315300-2950-4659-b12c-25b80f0aa68f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:35:20.377275 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:35:20.407735 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1460/2912] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:35:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:35:20.418323 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a266addc-5d2e-41a4-bb52-1d119fec7db5 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:20.419909 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a266addc-5d2e-41a4-bb52-1d119fec7db5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:35:20.478713 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a266addc-5d2e-41a4-bb52-1d119fec7db5 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:20.483357 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71831047-d3c1-4849-b161-4a3671c3186b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:20.483955 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71831047-d3c1-4849-b161-4a3671c3186b service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:20.484229 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71831047-d3c1-4849-b161-4a3671c3186b service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:20.484494 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71831047-d3c1-4849-b161-4a3671c3186b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:20.495095 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71831047-d3c1-4849-b161-4a3671c3186b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:20.496535 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71831047-d3c1-4849-b161-4a3671c3186b service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:20.506693 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1453/2913] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:35:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:35:20.813854 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c315300-2950-4659-b12c-25b80f0aa68f None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:20.815928 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-4c315300-2950-4659-b12c-25b80f0aa68f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:35:20.837494 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c315300-2950-4659-b12c-25b80f0aa68f None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:20.844922 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b9bdd7-fdf8-43b6-8b04-9b6ce3de411d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:20.845584 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b9bdd7-fdf8-43b6-8b04-9b6ce3de411d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:20.845852 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b9bdd7-fdf8-43b6-8b04-9b6ce3de411d admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:20.846138 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b9bdd7-fdf8-43b6-8b04-9b6ce3de411d admin admin] PATH_INFO: `/v3/projects/dea3dca0e3464a578a52cd619d5a338c` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:20.847386 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b9bdd7-fdf8-43b6-8b04-9b6ce3de411d admin admin] RBAC: Authorizing `identity:delete_project(project_id=dea3dca0e3464a578a52cd619d5a338c)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:20.849515 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b9bdd7-fdf8-43b6-8b04-9b6ce3de411d admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:20.905712 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e7b9bdd7-fdf8-43b6-8b04-9b6ce3de411d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dea3dca0e3464a578a52cd619d5a338c', 'request_id': 'req-e7b9bdd7-fdf8-43b6-8b04-9b6ce3de411d'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:20.914641 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e7b9bdd7-fdf8-43b6-8b04-9b6ce3de411d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dea3dca0e3464a578a52cd619d5a338c', 'request_id': 'req-e7b9bdd7-fdf8-43b6-8b04-9b6ce3de411d'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:20.924609 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e7b9bdd7-fdf8-43b6-8b04-9b6ce3de411d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dea3dca0e3464a578a52cd619d5a338c', 'request_id': 'req-e7b9bdd7-fdf8-43b6-8b04-9b6ce3de411d'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:20.937748 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e7b9bdd7-fdf8-43b6-8b04-9b6ce3de411d admin admin] The token cache is being invalidate because project dea3dca0e3464a578a52cd619d5a338c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:35:20.938119 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e7b9bdd7-fdf8-43b6-8b04-9b6ce3de411d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:20.939069 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e7b9bdd7-fdf8-43b6-8b04-9b6ce3de411d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:20.940851 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1461/2914] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:35:20 2022] DELETE /identity/v3/projects/dea3dca0e3464a578a52cd619d5a338c => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:35:20.945118 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71831047-d3c1-4849-b161-4a3671c3186b service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:20.955647 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71831047-d3c1-4849-b161-4a3671c3186b service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:21.026171 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df0e80e-64e3-4bbd-87d3-7e4927fcb86b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:21.026809 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df0e80e-64e3-4bbd-87d3-7e4927fcb86b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:21.027087 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df0e80e-64e3-4bbd-87d3-7e4927fcb86b admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:21.027362 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df0e80e-64e3-4bbd-87d3-7e4927fcb86b admin admin] PATH_INFO: `/v3/users/2ebd3aedbe4e413ab9e0a0754f5ea245` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:21.028572 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df0e80e-64e3-4bbd-87d3-7e4927fcb86b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ebd3aedbe4e413ab9e0a0754f5ea245)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:21.030550 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df0e80e-64e3-4bbd-87d3-7e4927fcb86b admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:21.078768 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-6df0e80e-64e3-4bbd-87d3-7e4927fcb86b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ebd3aedbe4e413ab9e0a0754f5ea245'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:21.079461 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-6df0e80e-64e3-4bbd-87d3-7e4927fcb86b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ebd3aedbe4e413ab9e0a0754f5ea245'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:21.085116 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:35:21.087151 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-6df0e80e-64e3-4bbd-87d3-7e4927fcb86b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ebd3aedbe4e413ab9e0a0754f5ea245'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:21.101676 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-6df0e80e-64e3-4bbd-87d3-7e4927fcb86b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ebd3aedbe4e413ab9e0a0754f5ea245'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:21.112173 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-6df0e80e-64e3-4bbd-87d3-7e4927fcb86b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ebd3aedbe4e413ab9e0a0754f5ea245'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:21.118968 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1454/2915] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:35:20 2022] DELETE /identity/v3/users/2ebd3aedbe4e413ab9e0a0754f5ea245 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:35:21.402319 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b9bdd7-fdf8-43b6-8b04-9b6ce3de411d admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:21.403796 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e7b9bdd7-fdf8-43b6-8b04-9b6ce3de411d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:35:21.417695 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b9bdd7-fdf8-43b6-8b04-9b6ce3de411d admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:21.476302 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a376107b-63c6-4e45-a611-5b3e256579bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:21.476976 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a376107b-63c6-4e45-a611-5b3e256579bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:21.477251 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a376107b-63c6-4e45-a611-5b3e256579bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:21.477527 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a376107b-63c6-4e45-a611-5b3e256579bf admin admin] PATH_INFO: `/v3/projects/29506b4457d24be0b89c366ecefe2577` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:21.478646 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a376107b-63c6-4e45-a611-5b3e256579bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=29506b4457d24be0b89c366ecefe2577)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:21.480573 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a376107b-63c6-4e45-a611-5b3e256579bf admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:21.528368 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a376107b-63c6-4e45-a611-5b3e256579bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29506b4457d24be0b89c366ecefe2577', 'request_id': 'req-a376107b-63c6-4e45-a611-5b3e256579bf'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:21.537565 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a376107b-63c6-4e45-a611-5b3e256579bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29506b4457d24be0b89c366ecefe2577', 'request_id': 'req-a376107b-63c6-4e45-a611-5b3e256579bf'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:21.540143 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:35:21.547002 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a376107b-63c6-4e45-a611-5b3e256579bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29506b4457d24be0b89c366ecefe2577', 'request_id': 'req-a376107b-63c6-4e45-a611-5b3e256579bf'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:21.560621 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a376107b-63c6-4e45-a611-5b3e256579bf admin admin] The token cache is being invalidate because project 29506b4457d24be0b89c366ecefe2577 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:35:21.560917 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a376107b-63c6-4e45-a611-5b3e256579bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:21.561636 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a376107b-63c6-4e45-a611-5b3e256579bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:21.563141 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1462/2916] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:35:21 2022] DELETE /identity/v3/projects/29506b4457d24be0b89c366ecefe2577 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:35:21.568350 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892cd9f1-f6da-4d8d-a3c6-df02888a23d9 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:21.568920 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892cd9f1-f6da-4d8d-a3c6-df02888a23d9 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:21.569031 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892cd9f1-f6da-4d8d-a3c6-df02888a23d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:21.589758 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-892cd9f1-f6da-4d8d-a3c6-df02888a23d9 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:35:21.590972 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-892cd9f1-f6da-4d8d-a3c6-df02888a23d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:35:21.653781 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1455/2917] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:35:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:35:21.662995 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b3f99-5d73-46f2-9ff0-0d6b40525cc6 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:21.663273 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b3f99-5d73-46f2-9ff0-0d6b40525cc6 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:21.663539 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b3f99-5d73-46f2-9ff0-0d6b40525cc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:21.684994 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-6e8b3f99-5d73-46f2-9ff0-0d6b40525cc6 None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:35:21.686182 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6e8b3f99-5d73-46f2-9ff0-0d6b40525cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:35:21.695409 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1463/2918] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:35:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:35:21.700088 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892cd9f1-f6da-4d8d-a3c6-df02888a23d9 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:21.710712 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892cd9f1-f6da-4d8d-a3c6-df02888a23d9 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:21.716708 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b1c304-4364-499c-8126-948bc819c908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:21.717380 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b1c304-4364-499c-8126-948bc819c908 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:21.717654 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b1c304-4364-499c-8126-948bc819c908 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:21.717914 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b1c304-4364-499c-8126-948bc819c908 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:21.718560 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b1c304-4364-499c-8126-948bc819c908 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:35:21.718853 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b1c304-4364-499c-8126-948bc819c908 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:21.720457 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b1c304-4364-499c-8126-948bc819c908 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:21.734929 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1456/2919] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 6 16:35:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:35:21.739706 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8b3f99-5d73-46f2-9ff0-0d6b40525cc6 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:21.752985 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8b3f99-5d73-46f2-9ff0-0d6b40525cc6 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:21.760155 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85e4589-94c8-48cf-a3b1-6c10ae46facb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:21.760783 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85e4589-94c8-48cf-a3b1-6c10ae46facb admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:21.761136 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85e4589-94c8-48cf-a3b1-6c10ae46facb admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:21.761404 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85e4589-94c8-48cf-a3b1-6c10ae46facb admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:21.762031 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85e4589-94c8-48cf-a3b1-6c10ae46facb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:21.763952 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85e4589-94c8-48cf-a3b1-6c10ae46facb admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:21.795470 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1464/2920] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:35:21 2022] POST /identity/v3/projects => generated 383 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:35:21.800007 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b1c304-4364-499c-8126-948bc819c908 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:21.810326 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b1c304-4364-499c-8126-948bc819c908 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:21.869013 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c98dc5-fc02-4809-aff4-4600b2b0a57a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:21.869641 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c98dc5-fc02-4809-aff4-4600b2b0a57a admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:21.869906 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c98dc5-fc02-4809-aff4-4600b2b0a57a admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:21.870167 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c98dc5-fc02-4809-aff4-4600b2b0a57a admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:21.870799 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c98dc5-fc02-4809-aff4-4600b2b0a57a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:21.872725 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c98dc5-fc02-4809-aff4-4600b2b0a57a admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:21.896713 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1457/2921] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:35:21 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:35:21.901268 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85e4589-94c8-48cf-a3b1-6c10ae46facb admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:21.913115 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85e4589-94c8-48cf-a3b1-6c10ae46facb admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:21.918461 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eddc12-b695-451b-b864-ee1303e83b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:21.919069 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eddc12-b695-451b-b864-ee1303e83b5b admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:21.919356 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eddc12-b695-451b-b864-ee1303e83b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:21.919629 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eddc12-b695-451b-b864-ee1303e83b5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:21.920262 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eddc12-b695-451b-b864-ee1303e83b5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:21.922085 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eddc12-b695-451b-b864-ee1303e83b5b admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:21.933562 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1465/2922] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:35:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:35:21.938422 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c98dc5-fc02-4809-aff4-4600b2b0a57a admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:21.954136 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c98dc5-fc02-4809-aff4-4600b2b0a57a admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:21.961307 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bcca08-80d5-4e62-86f0-36e189c0658a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:21.961914 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bcca08-80d5-4e62-86f0-36e189c0658a admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:21.962186 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bcca08-80d5-4e62-86f0-36e189c0658a admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:21.962452 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bcca08-80d5-4e62-86f0-36e189c0658a admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:21.963093 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bcca08-80d5-4e62-86f0-36e189c0658a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:21.964689 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bcca08-80d5-4e62-86f0-36e189c0658a admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:21.975248 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1458/2923] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:35:21 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:35:21.979711 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eddc12-b695-451b-b864-ee1303e83b5b admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:21.991610 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eddc12-b695-451b-b864-ee1303e83b5b admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:21.996618 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bb4430-3a5f-4daa-b329-e8f1513405c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:21.997316 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bb4430-3a5f-4daa-b329-e8f1513405c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:21.997593 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bb4430-3a5f-4daa-b329-e8f1513405c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:21.997867 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bb4430-3a5f-4daa-b329-e8f1513405c5 admin admin] PATH_INFO: `/v3/projects/029b92c5c9874a3ea34f3f4b0746f4b0/users/619dc7aca376433e96bdb152b2dd4b5c/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:22.013724 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bb4430-3a5f-4daa-b329-e8f1513405c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=029b92c5c9874a3ea34f3f4b0746f4b0, user_id=619dc7aca376433e96bdb152b2dd4b5c, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:22.016438 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bb4430-3a5f-4daa-b329-e8f1513405c5 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:22.030412 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1466/2924] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:35:21 2022] PUT /identity/v3/projects/029b92c5c9874a3ea34f3f4b0746f4b0/users/619dc7aca376433e96bdb152b2dd4b5c/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:35:22.040719 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1175dd-d019-4627-802d-c80548892615 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:22.041033 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1175dd-d019-4627-802d-c80548892615 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:22.041623 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1175dd-d019-4627-802d-c80548892615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:22.070348 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-1b1175dd-d019-4627-802d-c80548892615 None None] MFA Rules not processed for user `619dc7aca376433e96bdb152b2dd4b5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:35:22.071588 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-1b1175dd-d019-4627-802d-c80548892615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:35:22.098690 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:35:22.139403 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1459/2925] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:35:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:35:22.144254 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bb4430-3a5f-4daa-b329-e8f1513405c5 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:22.209866 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bb4430-3a5f-4daa-b329-e8f1513405c5 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:22.215078 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d3056b-e8d1-402e-a326-fe5a42e6e3b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:22.215691 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d3056b-e8d1-402e-a326-fe5a42e6e3b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:22.215971 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d3056b-e8d1-402e-a326-fe5a42e6e3b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:22.216238 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d3056b-e8d1-402e-a326-fe5a42e6e3b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:22.216901 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d3056b-e8d1-402e-a326-fe5a42e6e3b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:22.218844 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d3056b-e8d1-402e-a326-fe5a42e6e3b6 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:22.250551 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1467/2926] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:35:22 2022] POST /identity/v3/projects => generated 381 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:35:22.251780 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1175dd-d019-4627-802d-c80548892615 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:22.263285 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1175dd-d019-4627-802d-c80548892615 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:22.327996 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2adfdf-363c-44c9-8b25-a80aa86207bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:22.328644 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2adfdf-363c-44c9-8b25-a80aa86207bf admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:22.328980 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2adfdf-363c-44c9-8b25-a80aa86207bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:22.329255 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2adfdf-363c-44c9-8b25-a80aa86207bf admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:22.329922 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2adfdf-363c-44c9-8b25-a80aa86207bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:22.331877 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2adfdf-363c-44c9-8b25-a80aa86207bf admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:22.356313 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1460/2927] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:35:22 2022] POST /identity/v3/users => generated 297 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:35:22.360687 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d3056b-e8d1-402e-a326-fe5a42e6e3b6 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:22.371800 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d3056b-e8d1-402e-a326-fe5a42e6e3b6 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:22.376550 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e24ea6a-e609-4de0-a58e-bb34a40b643f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:22.377189 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e24ea6a-e609-4de0-a58e-bb34a40b643f admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:22.377439 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e24ea6a-e609-4de0-a58e-bb34a40b643f admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:22.377706 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e24ea6a-e609-4de0-a58e-bb34a40b643f admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:22.378355 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e24ea6a-e609-4de0-a58e-bb34a40b643f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:22.379973 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e24ea6a-e609-4de0-a58e-bb34a40b643f admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:22.391392 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1468/2928] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:35:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:35:22.396328 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2adfdf-363c-44c9-8b25-a80aa86207bf admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:22.407417 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2adfdf-363c-44c9-8b25-a80aa86207bf admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:22.413346 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f173a81-f490-40b7-8d96-f00f2df40e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:22.414016 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f173a81-f490-40b7-8d96-f00f2df40e4a admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:22.414291 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f173a81-f490-40b7-8d96-f00f2df40e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:22.414561 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f173a81-f490-40b7-8d96-f00f2df40e4a admin admin] PATH_INFO: `/v3/projects/42b8a9f0e2b144f4839a9d4089833e77/users/9b82d51925ae4e42808891ca2f184b48/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:22.432477 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f173a81-f490-40b7-8d96-f00f2df40e4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=42b8a9f0e2b144f4839a9d4089833e77, user_id=9b82d51925ae4e42808891ca2f184b48, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:22.435281 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f173a81-f490-40b7-8d96-f00f2df40e4a admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:22.450544 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1461/2929] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:35:22 2022] PUT /identity/v3/projects/42b8a9f0e2b144f4839a9d4089833e77/users/9b82d51925ae4e42808891ca2f184b48/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:35:22.454576 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e24ea6a-e609-4de0-a58e-bb34a40b643f admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:22.465507 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e24ea6a-e609-4de0-a58e-bb34a40b643f admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:22.527934 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27c2f63-4db4-41f2-a322-38a42c1daaa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:22.528542 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27c2f63-4db4-41f2-a322-38a42c1daaa5 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:22.528826 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27c2f63-4db4-41f2-a322-38a42c1daaa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:22.529117 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27c2f63-4db4-41f2-a322-38a42c1daaa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:22.529759 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27c2f63-4db4-41f2-a322-38a42c1daaa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:22.531396 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27c2f63-4db4-41f2-a322-38a42c1daaa5 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:22.532883 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:35:22.545376 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1469/2930] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:35:22 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 16:35:22.548577 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f173a81-f490-40b7-8d96-f00f2df40e4a admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:22.560120 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f173a81-f490-40b7-8d96-f00f2df40e4a admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:22.565601 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4f168c-b924-4337-b73e-7e1e386fe5ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:22.566269 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4f168c-b924-4337-b73e-7e1e386fe5ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:22.566541 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4f168c-b924-4337-b73e-7e1e386fe5ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:22.566812 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4f168c-b924-4337-b73e-7e1e386fe5ed admin admin] PATH_INFO: `/v3/projects/42b8a9f0e2b144f4839a9d4089833e77/users/9b82d51925ae4e42808891ca2f184b48/roles/edaabdea8be84a7980fb2c651fe52f63` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:22.568791 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4f168c-b924-4337-b73e-7e1e386fe5ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=42b8a9f0e2b144f4839a9d4089833e77, user_id=9b82d51925ae4e42808891ca2f184b48, role_id=edaabdea8be84a7980fb2c651fe52f63)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:22.571509 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4f168c-b924-4337-b73e-7e1e386fe5ed admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:22.582659 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1462/2931] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:35:22 2022] PUT /identity/v3/projects/42b8a9f0e2b144f4839a9d4089833e77/users/9b82d51925ae4e42808891ca2f184b48/roles/edaabdea8be84a7980fb2c651fe52f63 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:35:22.593337 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2397350-1456-4f73-83f0-20d50908f956 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:22.593626 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2397350-1456-4f73-83f0-20d50908f956 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:22.593922 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2397350-1456-4f73-83f0-20d50908f956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:22.633951 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-d2397350-1456-4f73-83f0-20d50908f956 None None] MFA Rules not processed for user `9b82d51925ae4e42808891ca2f184b48`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:35:22.635189 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d2397350-1456-4f73-83f0-20d50908f956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:35:22.711988 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1470/2932] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:35:22 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:35:22.726828 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4f168c-b924-4337-b73e-7e1e386fe5ed admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:22.728793 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5a4f168c-b924-4337-b73e-7e1e386fe5ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:35:22.802870 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4f168c-b924-4337-b73e-7e1e386fe5ed admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:22.807344 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db3933b-70da-4fde-b772-4ee84194930d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:22.807977 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db3933b-70da-4fde-b772-4ee84194930d service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:22.808256 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db3933b-70da-4fde-b772-4ee84194930d service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:22.808500 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db3933b-70da-4fde-b772-4ee84194930d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:22.819240 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db3933b-70da-4fde-b772-4ee84194930d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:22.820706 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db3933b-70da-4fde-b772-4ee84194930d service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:22.832337 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1463/2933] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:35:22 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:35:22.915887 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2397350-1456-4f73-83f0-20d50908f956 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:22.927484 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2397350-1456-4f73-83f0-20d50908f956 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:22.936889 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e9561e-239c-4c8b-a1ca-c1312cdd8077 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:22.937503 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e9561e-239c-4c8b-a1ca-c1312cdd8077 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:22.937776 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e9561e-239c-4c8b-a1ca-c1312cdd8077 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:22.938042 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e9561e-239c-4c8b-a1ca-c1312cdd8077 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:23.000314 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e9561e-239c-4c8b-a1ca-c1312cdd8077 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:23.001712 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e9561e-239c-4c8b-a1ca-c1312cdd8077 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:23.011374 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1471/2934] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:35:22 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:35:23.309550 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db3933b-70da-4fde-b772-4ee84194930d service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:23.334614 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:35:23.372633 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db3933b-70da-4fde-b772-4ee84194930d service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:23.377505 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650b70cd-acc0-4c6f-bb10-a0c73da308a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:23.378124 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b70cd-acc0-4c6f-bb10-a0c73da308a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:23.378396 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b70cd-acc0-4c6f-bb10-a0c73da308a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:23.378666 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b70cd-acc0-4c6f-bb10-a0c73da308a6 admin admin] PATH_INFO: `/v3/users/619dc7aca376433e96bdb152b2dd4b5c` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:23.379805 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650b70cd-acc0-4c6f-bb10-a0c73da308a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=619dc7aca376433e96bdb152b2dd4b5c)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:23.381848 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650b70cd-acc0-4c6f-bb10-a0c73da308a6 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:23.434664 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-650b70cd-acc0-4c6f-bb10-a0c73da308a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '619dc7aca376433e96bdb152b2dd4b5c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:23.435325 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-650b70cd-acc0-4c6f-bb10-a0c73da308a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '619dc7aca376433e96bdb152b2dd4b5c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:23.442202 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-650b70cd-acc0-4c6f-bb10-a0c73da308a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '619dc7aca376433e96bdb152b2dd4b5c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:23.453879 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-650b70cd-acc0-4c6f-bb10-a0c73da308a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '619dc7aca376433e96bdb152b2dd4b5c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:23.462958 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-650b70cd-acc0-4c6f-bb10-a0c73da308a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '619dc7aca376433e96bdb152b2dd4b5c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:23.469213 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1464/2935] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:35:23 2022] DELETE /identity/v3/users/619dc7aca376433e96bdb152b2dd4b5c => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:35:23.473852 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752dd897-19eb-4179-a644-84b77eb8904f None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:23.474322 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752dd897-19eb-4179-a644-84b77eb8904f None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:23.474469 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752dd897-19eb-4179-a644-84b77eb8904f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:23.495077 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-752dd897-19eb-4179-a644-84b77eb8904f None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:35:23.496325 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-752dd897-19eb-4179-a644-84b77eb8904f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:35:23.558569 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1472/2936] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:35:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:35:23.568317 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650b70cd-acc0-4c6f-bb10-a0c73da308a6 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:23.569905 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-650b70cd-acc0-4c6f-bb10-a0c73da308a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:35:23.634464 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650b70cd-acc0-4c6f-bb10-a0c73da308a6 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:23.639061 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df3a492-48e5-4cea-bbb9-a871835d4c76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:23.639702 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df3a492-48e5-4cea-bbb9-a871835d4c76 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:23.639983 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df3a492-48e5-4cea-bbb9-a871835d4c76 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:23.640260 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df3a492-48e5-4cea-bbb9-a871835d4c76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:23.650706 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df3a492-48e5-4cea-bbb9-a871835d4c76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:23.652146 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df3a492-48e5-4cea-bbb9-a871835d4c76 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:23.661872 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1465/2937] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:35:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:35:23.929602 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752dd897-19eb-4179-a644-84b77eb8904f None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:23.931151 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-752dd897-19eb-4179-a644-84b77eb8904f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:35:23.946707 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752dd897-19eb-4179-a644-84b77eb8904f None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:23.952237 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bf9ffd-c228-4e5e-998a-0ed28c9b7a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:23.952902 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bf9ffd-c228-4e5e-998a-0ed28c9b7a08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:23.953177 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bf9ffd-c228-4e5e-998a-0ed28c9b7a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:23.953446 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bf9ffd-c228-4e5e-998a-0ed28c9b7a08 admin admin] PATH_INFO: `/v3/projects/029b92c5c9874a3ea34f3f4b0746f4b0` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:23.954976 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bf9ffd-c228-4e5e-998a-0ed28c9b7a08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=029b92c5c9874a3ea34f3f4b0746f4b0)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:23.957114 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bf9ffd-c228-4e5e-998a-0ed28c9b7a08 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:23.974869 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:35:24.007822 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-72bf9ffd-c228-4e5e-998a-0ed28c9b7a08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '029b92c5c9874a3ea34f3f4b0746f4b0', 'request_id': 'req-72bf9ffd-c228-4e5e-998a-0ed28c9b7a08'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:24.013313 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-72bf9ffd-c228-4e5e-998a-0ed28c9b7a08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '029b92c5c9874a3ea34f3f4b0746f4b0', 'request_id': 'req-72bf9ffd-c228-4e5e-998a-0ed28c9b7a08'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:24.022130 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-72bf9ffd-c228-4e5e-998a-0ed28c9b7a08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '029b92c5c9874a3ea34f3f4b0746f4b0', 'request_id': 'req-72bf9ffd-c228-4e5e-998a-0ed28c9b7a08'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:24.033925 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-72bf9ffd-c228-4e5e-998a-0ed28c9b7a08 admin admin] The token cache is being invalidate because project 029b92c5c9874a3ea34f3f4b0746f4b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:35:24.034202 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-72bf9ffd-c228-4e5e-998a-0ed28c9b7a08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:24.034864 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-72bf9ffd-c228-4e5e-998a-0ed28c9b7a08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:24.036394 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1473/2938] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:35:23 2022] DELETE /identity/v3/projects/029b92c5c9874a3ea34f3f4b0746f4b0 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:35:24.040429 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df3a492-48e5-4cea-bbb9-a871835d4c76 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:24.050781 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df3a492-48e5-4cea-bbb9-a871835d4c76 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:24.094871 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:35:24.118354 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbeb163-ef45-443b-8c20-bcb0056e903d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:24.118970 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbeb163-ef45-443b-8c20-bcb0056e903d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:24.119245 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbeb163-ef45-443b-8c20-bcb0056e903d admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:24.119672 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbeb163-ef45-443b-8c20-bcb0056e903d admin admin] PATH_INFO: `/v3/users/9b82d51925ae4e42808891ca2f184b48` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:24.120866 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbeb163-ef45-443b-8c20-bcb0056e903d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b82d51925ae4e42808891ca2f184b48)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:24.122782 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbeb163-ef45-443b-8c20-bcb0056e903d admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:24.173385 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-ccbeb163-ef45-443b-8c20-bcb0056e903d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b82d51925ae4e42808891ca2f184b48'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:24.174107 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-ccbeb163-ef45-443b-8c20-bcb0056e903d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b82d51925ae4e42808891ca2f184b48'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:24.182880 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-ccbeb163-ef45-443b-8c20-bcb0056e903d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b82d51925ae4e42808891ca2f184b48'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:24.194394 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-ccbeb163-ef45-443b-8c20-bcb0056e903d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b82d51925ae4e42808891ca2f184b48'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:24.202906 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-ccbeb163-ef45-443b-8c20-bcb0056e903d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b82d51925ae4e42808891ca2f184b48'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:24.209688 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1466/2939] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:35:24 2022] DELETE /identity/v3/users/9b82d51925ae4e42808891ca2f184b48 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:35:24.511494 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bf9ffd-c228-4e5e-998a-0ed28c9b7a08 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:24.517231 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-72bf9ffd-c228-4e5e-998a-0ed28c9b7a08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:35:24.532500 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bf9ffd-c228-4e5e-998a-0ed28c9b7a08 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:24.593171 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decf2390-9bd3-4c07-9472-3a57e2f23713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:24.593809 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decf2390-9bd3-4c07-9472-3a57e2f23713 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:24.594086 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decf2390-9bd3-4c07-9472-3a57e2f23713 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:24.594350 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decf2390-9bd3-4c07-9472-3a57e2f23713 admin admin] PATH_INFO: `/v3/projects/42b8a9f0e2b144f4839a9d4089833e77` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:24.596192 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decf2390-9bd3-4c07-9472-3a57e2f23713 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42b8a9f0e2b144f4839a9d4089833e77)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:24.598142 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decf2390-9bd3-4c07-9472-3a57e2f23713 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:24.647989 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-decf2390-9bd3-4c07-9472-3a57e2f23713 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42b8a9f0e2b144f4839a9d4089833e77', 'request_id': 'req-decf2390-9bd3-4c07-9472-3a57e2f23713'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:24.653695 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-decf2390-9bd3-4c07-9472-3a57e2f23713 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42b8a9f0e2b144f4839a9d4089833e77', 'request_id': 'req-decf2390-9bd3-4c07-9472-3a57e2f23713'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:24.662372 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-decf2390-9bd3-4c07-9472-3a57e2f23713 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42b8a9f0e2b144f4839a9d4089833e77', 'request_id': 'req-decf2390-9bd3-4c07-9472-3a57e2f23713'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:24.679568 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-decf2390-9bd3-4c07-9472-3a57e2f23713 admin admin] The token cache is being invalidate because project 42b8a9f0e2b144f4839a9d4089833e77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:35:24.679895 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-decf2390-9bd3-4c07-9472-3a57e2f23713 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:24.680944 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-decf2390-9bd3-4c07-9472-3a57e2f23713 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:24.683174 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1474/2940] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:35:24 2022] DELETE /identity/v3/projects/42b8a9f0e2b144f4839a9d4089833e77 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:35:24.690643 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db4e1b-d130-4631-b094-9bfe2a077d0c None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:24.691165 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db4e1b-d130-4631-b094-9bfe2a077d0c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:24.691271 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db4e1b-d130-4631-b094-9bfe2a077d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:24.725309 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-d9db4e1b-d130-4631-b094-9bfe2a077d0c None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:35:24.726501 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d9db4e1b-d130-4631-b094-9bfe2a077d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:35:24.790337 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1467/2941] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:35:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:35:24.798297 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7adad8a-c23c-4be1-8f83-e957d359ad7a None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:24.798584 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7adad8a-c23c-4be1-8f83-e957d359ad7a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:24.798859 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7adad8a-c23c-4be1-8f83-e957d359ad7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:24.819792 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-d7adad8a-c23c-4be1-8f83-e957d359ad7a None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:35:24.821058 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d7adad8a-c23c-4be1-8f83-e957d359ad7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:35:24.829086 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1475/2942] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:35:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:35:24.833549 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9db4e1b-d130-4631-b094-9bfe2a077d0c None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:24.849520 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9db4e1b-d130-4631-b094-9bfe2a077d0c None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:24.858143 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdabd90-a269-4e08-841c-2a0ba20c9fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:24.859084 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdabd90-a269-4e08-841c-2a0ba20c9fe5 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:24.859559 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdabd90-a269-4e08-841c-2a0ba20c9fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:24.860000 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdabd90-a269-4e08-841c-2a0ba20c9fe5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:24.861133 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdabd90-a269-4e08-841c-2a0ba20c9fe5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:35:24.861655 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdabd90-a269-4e08-841c-2a0ba20c9fe5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:24.864658 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdabd90-a269-4e08-841c-2a0ba20c9fe5 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:24.885158 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1468/2943] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 6 16:35:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:35:24.889929 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7adad8a-c23c-4be1-8f83-e957d359ad7a None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:24.899921 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7adad8a-c23c-4be1-8f83-e957d359ad7a None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:24.905938 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7adf2d-6bbb-40b0-8561-ec1ff435bd22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:24.906558 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7adf2d-6bbb-40b0-8561-ec1ff435bd22 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:24.906859 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7adf2d-6bbb-40b0-8561-ec1ff435bd22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:24.907128 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7adf2d-6bbb-40b0-8561-ec1ff435bd22 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:24.907758 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7adf2d-6bbb-40b0-8561-ec1ff435bd22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:24.909742 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7adf2d-6bbb-40b0-8561-ec1ff435bd22 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:24.944199 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1476/2944] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 6 16:35:24 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:35:24.948814 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdabd90-a269-4e08-841c-2a0ba20c9fe5 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:24.959262 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdabd90-a269-4e08-841c-2a0ba20c9fe5 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:25.006405 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:35:25.018165 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca06fea-1461-48c3-9b30-6192b00c671f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:25.018805 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca06fea-1461-48c3-9b30-6192b00c671f admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:25.019067 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca06fea-1461-48c3-9b30-6192b00c671f admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:25.019344 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca06fea-1461-48c3-9b30-6192b00c671f admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:25.019994 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca06fea-1461-48c3-9b30-6192b00c671f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:25.022301 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca06fea-1461-48c3-9b30-6192b00c671f admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:25.045904 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1469/2945] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 6 16:35:24 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:35:25.050441 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7adf2d-6bbb-40b0-8561-ec1ff435bd22 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:25.057664 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:35:25.061578 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7adf2d-6bbb-40b0-8561-ec1ff435bd22 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:25.068157 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e784cac9-1f2d-4cea-8313-9a1fb93d4795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:25.069085 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e784cac9-1f2d-4cea-8313-9a1fb93d4795 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:25.069411 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e784cac9-1f2d-4cea-8313-9a1fb93d4795 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:25.069834 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e784cac9-1f2d-4cea-8313-9a1fb93d4795 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:25.070532 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e784cac9-1f2d-4cea-8313-9a1fb93d4795 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:25.072349 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e784cac9-1f2d-4cea-8313-9a1fb93d4795 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:25.082979 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1477/2946] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:35:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:35:25.087998 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca06fea-1461-48c3-9b30-6192b00c671f admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:25.100676 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca06fea-1461-48c3-9b30-6192b00c671f admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:25.106610 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e4947f-1e8e-4180-8012-5aa8b0fba24b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:25.107222 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e4947f-1e8e-4180-8012-5aa8b0fba24b admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:25.107493 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e4947f-1e8e-4180-8012-5aa8b0fba24b admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:25.107759 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e4947f-1e8e-4180-8012-5aa8b0fba24b admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:25.108399 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e4947f-1e8e-4180-8012-5aa8b0fba24b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:25.110587 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e4947f-1e8e-4180-8012-5aa8b0fba24b admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:25.122336 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1470/2947] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 6 16:35:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:35:25.126883 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e784cac9-1f2d-4cea-8313-9a1fb93d4795 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:25.137781 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e784cac9-1f2d-4cea-8313-9a1fb93d4795 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:25.142922 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75c98df-00d3-4cd3-b282-89dbafd4a287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:25.143588 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75c98df-00d3-4cd3-b282-89dbafd4a287 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:25.143862 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75c98df-00d3-4cd3-b282-89dbafd4a287 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:25.144137 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75c98df-00d3-4cd3-b282-89dbafd4a287 admin admin] PATH_INFO: `/v3/projects/731cebdf147d4ab9925be0740b86bca9/users/83433394e7474038b5485cc42b6fa2b6/roles/15252b0b89894c1e8a6bd9a98b1c3ab7` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:25.159978 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75c98df-00d3-4cd3-b282-89dbafd4a287 admin admin] RBAC: Authorizing `identity:create_grant(project_id=731cebdf147d4ab9925be0740b86bca9, user_id=83433394e7474038b5485cc42b6fa2b6, role_id=15252b0b89894c1e8a6bd9a98b1c3ab7)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:25.162725 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75c98df-00d3-4cd3-b282-89dbafd4a287 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:25.177965 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1478/2948] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 6 16:35:25 2022] PUT /identity/v3/projects/731cebdf147d4ab9925be0740b86bca9/users/83433394e7474038b5485cc42b6fa2b6/roles/15252b0b89894c1e8a6bd9a98b1c3ab7 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:35:25.188414 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaf9002-aa22-4a8d-aec1-5b183c2ad318 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:25.188695 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaf9002-aa22-4a8d-aec1-5b183c2ad318 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:25.189038 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaf9002-aa22-4a8d-aec1-5b183c2ad318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:25.223189 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-feaf9002-aa22-4a8d-aec1-5b183c2ad318 None None] MFA Rules not processed for user `83433394e7474038b5485cc42b6fa2b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:35:25.224466 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-feaf9002-aa22-4a8d-aec1-5b183c2ad318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:35:25.291963 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1471/2949] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:35:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:35:25.303786 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75c98df-00d3-4cd3-b282-89dbafd4a287 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:25.305596 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d75c98df-00d3-4cd3-b282-89dbafd4a287 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:35:25.372652 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75c98df-00d3-4cd3-b282-89dbafd4a287 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:25.376607 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c8da35-2f11-4727-848d-1f05b0374e23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b05748e4284153b0728164cd508dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:25.377296 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c8da35-2f11-4727-848d-1f05b0374e23 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:25.377545 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c8da35-2f11-4727-848d-1f05b0374e23 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:25.377846 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c8da35-2f11-4727-848d-1f05b0374e23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:25.386933 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c8da35-2f11-4727-848d-1f05b0374e23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:25.388379 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c8da35-2f11-4727-848d-1f05b0374e23 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:25.397521 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1479/2950] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:35:25 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:35:25.418169 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaf9002-aa22-4a8d-aec1-5b183c2ad318 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:25.483747 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaf9002-aa22-4a8d-aec1-5b183c2ad318 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:25.489064 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfa5499-7cde-4c1c-9e6f-885a4a5b392e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:25.489695 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfa5499-7cde-4c1c-9e6f-885a4a5b392e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:25.490002 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfa5499-7cde-4c1c-9e6f-885a4a5b392e admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:25.490251 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfa5499-7cde-4c1c-9e6f-885a4a5b392e admin admin] PATH_INFO: `/v3/users/83433394e7474038b5485cc42b6fa2b6` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:25.491415 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfa5499-7cde-4c1c-9e6f-885a4a5b392e admin admin] RBAC: Authorizing `identity:delete_user(user_id=83433394e7474038b5485cc42b6fa2b6)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:25.493372 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfa5499-7cde-4c1c-9e6f-885a4a5b392e admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:25.542940 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-fbfa5499-7cde-4c1c-9e6f-885a4a5b392e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83433394e7474038b5485cc42b6fa2b6'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:25.543709 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-fbfa5499-7cde-4c1c-9e6f-885a4a5b392e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83433394e7474038b5485cc42b6fa2b6'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:25.551415 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-fbfa5499-7cde-4c1c-9e6f-885a4a5b392e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83433394e7474038b5485cc42b6fa2b6'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:25.556016 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:35:25.565353 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-fbfa5499-7cde-4c1c-9e6f-885a4a5b392e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83433394e7474038b5485cc42b6fa2b6'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:25.573186 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-fbfa5499-7cde-4c1c-9e6f-885a4a5b392e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83433394e7474038b5485cc42b6fa2b6'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:25.579301 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1472/2951] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 6 16:35:25 2022] DELETE /identity/v3/users/83433394e7474038b5485cc42b6fa2b6 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:35:25.584262 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede31ce2-1ce9-410e-8daf-2a22fa1aadfe None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:25.584550 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede31ce2-1ce9-410e-8daf-2a22fa1aadfe None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:25.584881 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede31ce2-1ce9-410e-8daf-2a22fa1aadfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:25.604939 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-ede31ce2-1ce9-410e-8daf-2a22fa1aadfe None None] MFA Rules not processed for user `824c9880f41b4962b83243abf75b8162`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:35:25.606141 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ede31ce2-1ce9-410e-8daf-2a22fa1aadfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:35:25.669998 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1480/2952] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 6 16:35:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:35:25.680097 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfa5499-7cde-4c1c-9e6f-885a4a5b392e admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:25.681565 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-fbfa5499-7cde-4c1c-9e6f-885a4a5b392e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:35:25.740863 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfa5499-7cde-4c1c-9e6f-885a4a5b392e admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:25.745459 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b83c199-daa4-48e7-9241-7423e585268a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10565aefeaaa40549ed38bb9e0f4b189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:25.746083 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b83c199-daa4-48e7-9241-7423e585268a service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:25.746356 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b83c199-daa4-48e7-9241-7423e585268a service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:25.746621 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b83c199-daa4-48e7-9241-7423e585268a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:25.757526 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b83c199-daa4-48e7-9241-7423e585268a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:25.759028 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b83c199-daa4-48e7-9241-7423e585268a service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:25.769657 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1473/2953] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:35:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:35:26.057183 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede31ce2-1ce9-410e-8daf-2a22fa1aadfe None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:26.058708 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ede31ce2-1ce9-410e-8daf-2a22fa1aadfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:35:26.071676 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede31ce2-1ce9-410e-8daf-2a22fa1aadfe None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:26.077149 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a862f3b-8875-4da9-8375-36dadc6a86f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824c9880f41b4962b83243abf75b8162', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad262a5a8a84066b8eba54c7ea32d39', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:26.077793 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a862f3b-8875-4da9-8375-36dadc6a86f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:26.078072 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a862f3b-8875-4da9-8375-36dadc6a86f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:26.078445 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a862f3b-8875-4da9-8375-36dadc6a86f1 admin admin] PATH_INFO: `/v3/projects/731cebdf147d4ab9925be0740b86bca9` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:26.079948 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a862f3b-8875-4da9-8375-36dadc6a86f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=731cebdf147d4ab9925be0740b86bca9)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:26.081961 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a862f3b-8875-4da9-8375-36dadc6a86f1 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:26.139687 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9a862f3b-8875-4da9-8375-36dadc6a86f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '731cebdf147d4ab9925be0740b86bca9', 'request_id': 'req-9a862f3b-8875-4da9-8375-36dadc6a86f1'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:26.145585 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9a862f3b-8875-4da9-8375-36dadc6a86f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '731cebdf147d4ab9925be0740b86bca9', 'request_id': 'req-9a862f3b-8875-4da9-8375-36dadc6a86f1'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:26.154784 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9a862f3b-8875-4da9-8375-36dadc6a86f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '731cebdf147d4ab9925be0740b86bca9', 'request_id': 'req-9a862f3b-8875-4da9-8375-36dadc6a86f1'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:26.156907 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:35:26.168468 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9a862f3b-8875-4da9-8375-36dadc6a86f1 admin admin] The token cache is being invalidate because project 731cebdf147d4ab9925be0740b86bca9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:35:26.168868 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9a862f3b-8875-4da9-8375-36dadc6a86f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:26.169618 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9a862f3b-8875-4da9-8375-36dadc6a86f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:26.171204 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1481/2954] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 6 16:35:26 2022] DELETE /identity/v3/projects/731cebdf147d4ab9925be0740b86bca9 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:35:35.764951 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:DELETE=3,keystone:SELECT=32,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:35:36.171852 dv-cinder-852210-10-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:40:09.856308 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b83c199-daa4-48e7-9241-7423e585268a service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:40:09.858050 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0b83c199-daa4-48e7-9241-7423e585268a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:40:09.931888 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b83c199-daa4-48e7-9241-7423e585268a service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:40:09.992790 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef13f00e-d3ab-4558-a22d-a76028e36570 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71c2912a44f04bd983b4d4112dbee734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '128d210e8dad48ac921ed4cb1d444ceb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:40:09.993425 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef13f00e-d3ab-4558-a22d-a76028e36570 service placement] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:40:09.993703 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef13f00e-d3ab-4558-a22d-a76028e36570 service placement] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:40:09.993976 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef13f00e-d3ab-4558-a22d-a76028e36570 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:40:09.995976 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ef13f00e-d3ab-4558-a22d-a76028e36570 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:40:10.007503 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef13f00e-d3ab-4558-a22d-a76028e36570 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:40:10.009132 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef13f00e-d3ab-4558-a22d-a76028e36570 service placement] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:40:10.019014 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1474/2955] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 6 16:40:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:40:20.014768 dv-cinder-852210-10-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=58 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}